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Executive Summary 


Executive Summary 


This report presents the methods and findings of a qualitative study of nonrespondent 
addresses to the 2019 administration of the National Household Education Surveys Program 
(NHES:2019). The overarching goal of the study was to better understand the drivers of 
nonresponse to the NHES and to provide additional, actionable information on how to 
combat this growing problem. The study included two components. 


e In-depth interviews. Eighty-five qualitative interviews were conducted with 
individuals whose household had not responded to the NHES as of the final screener- 
phase mailing. The interviews lasted approximately 90 minutes, generally took place 
in the interviewee’s home, and followed a semistructured interview protocol 
consisting of eight domains hypothesized to be drivers of nonresponse (e.g., 
understanding of surveys, privacy concerns, attitudes toward government). 


e Address observations. Field staff also conducted 760 observations of addresses that 
were either (1) nonrespondents as of the final screener-phase mailing or (2) had 
inconsistent undeliverable-as-addressed (UAA) outcomes for the screener-phase 
mailings (that is, JAA outcomes for some but not all the mailings). After locating the 
sampled address, staff observed the exterior. They completed an observation form 
with both forced-choice and open-ended items that either (1) collected address 
characteristics not currently available on the NHES sampling frame or (2) could be 
used to assess the quality of the information available on the sampling frame. 


The most notable findings from the study are outlined below. More detailed findings and 
considerations for future NHES administrations can be found in chapters 3 through 8. In 
reviewing the study findings and conclusions, it is important to remember that this is a 
qualitative research study. It was not designed to be nationally representative of all NHES 
nonrespondents. 


Nonrespondent Characteristics and Attitudes 


Both the interviews and address observations were designed to explore the characteristics 
and attitudes of the households that do not respond to the NHES screener-phase mailings. 


e Interview themes. Due to the conversational nature of the interviews, a wide range 
of topics were discussed. Key themes that appear relevant to response decisions for 
the NHES include: (1) Many interview participants felt their lives were very 
busy. More than half specifically talked about being extremely busy, and many 
worked long hours. (2) Most of those who shared their views on the federal 
government had negative views, voicing a variety of concerns such as the belief that 
government is too intrusive, and several expressed concerns about government 
access to and collection of information. (3) Almost all participants discussed privacy 
at some point during the interview, although how they defined privacy and their level 
of concern about it varied. Many believed that there was no such thing as privacy— 
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their information was already freely available—while one in six were extremely 
concerned about privacy. (4) Almost all participants believed that education was 
extremely important, regardless of whether they had school-age children. 


Address observation findings. There was considerable variation in the frequency 
with which observers identified evidence of the characteristics and attitudes of 
interest. Privacy or security concerns (e.g., surveillance cameras), interest in outdoor 
living, and outdoor decor were the most commonly observed (ranging from 25 to 31 
percent of addresses). Indicators of other characteristics or interests were observed 
for less than 10 percent of addresses, suggesting that most households do not place 
items outside of the home that shed light on their values or interests. This was 
particularly true for apartments. 


Receiving and Processing Mail 


Interview participants’ comments related to receiving and processing mail shed light on the 
challenges associated with conducting a mail-based, self-administered survey. 


Receiving mail. While many interview participants checked their mail daily, those 
who did not do so every day tended to check their mail only when they were expecting 
certain items to be delivered. Some renters noted that the property owner used their 
address as his or her permanent address; when mail was addressed to the household 
(as is done in the NHES mailings), they assumed it was for the owner. About a third of 
participants discussed challenges with mail delivery, such as receiving mail 
addressed to a different address or not receiving expected mail items. Eight percent 
of the observed addresses did not have a mailbox or slot clearly in view. 


Processing mail. Unless they tended to open all or most of their mail, interview 
participants took numerous cues from envelopes into account when deciding how to 
sort and process mail, such as whether it was addressed to them personally, the 
degree of officialness, and the sender. These decisions seemed to be embedded in the 
context of participants’ daily lives, suggesting they are likely to change from mailing 
to mailing due to other demands on their attention and time. 


Experiences with and Attitudes Toward Surveys 


Most of the interview participants had completed a survey in the past, but their opinions 
about survey participation were varied. A review of materials used in the screener phase of 
NHES:2019 identified several themes in interview participants’ recall of and reaction to the 
mailings. 


Recall of the mailings. There was considerable variation in the point at which 
interview participants broke off from responding. About a quarter did not remember 
receiving any of the NHES screener mailings. About a fifth of those who remembered 
a mailing did not open it. Among those who opened at least one mailing, over half 
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discarded the mailings or actively decided not to respond, about a fifth saved the 
mailings but never ended up responding, and about a quarter responded to the 
survey. 


Reactions to the mailings. There were three main features of the mailings that 
influenced interview participants’ reaction to them: (1) the perceived importance of 
the mailings (e.g., government affiliation, FedEx delivery), (2) the personal relevance 
of the topic of education, and (3) the perceived burden or intrusiveness of the request. 
For some participants, factors unrelated to the NHES design affected their response 
(e.g., busyness, a general rule not to do any surveys). No single driver of nonresponse 
was mentioned by every participant; moreover, for several topics, different 
subgroups of participants had conflicting opinions. 


Nonrespondent Typologies 


The interview participants were classified on several factors representing the themes 
discussed above, and each participant was placed into a group based on these classifications. 
These typology groups provide a starting point for understanding the primary drivers of 
nonresponse and for considering potential changes that could be made to the NHES 
materials or design to address those issues. 


i 


Late respondents. This group included all interview participants whose household 
responded to the NHES after the fourth screener package. What kept most of them 
from responding earlier seems to be related to busyness. 


. Not enough time. Everyone in this group described themselves as being very busy; 


many had extreme demands on their time, and about 1 in 3 talked about being 
completely exhausted. Some explicitly said that they did not have time to take 
surveys. 


Negative attitudes toward the federal government. These participants had 
negative attitudes toward the federal government. They either wanted as little 
interaction with the government as possible or believed the government was corrupt 
and could not be trusted. 


Federal government already has my information. These participants did not see 
the purpose of completing the NHES. They believed the federal government already 
had access to their data, and it should make use of that before asking them to complete 
a survey. 


Not relevant to me. Everyone in this group believed K-12 education was not 
relevant to them. All households but one did not have children living in them. 


Multiple barriers. Participants in this group reported experiencing multiple barriers 
to completing the NHES. They each had a combination of factors that seemed to 
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influence their lack of response. They would likely be the hardest to convert to 
respondents. 


Less likely to recall NHES mailings. These participants checked their mail 
frequently, but just under half remembered at least one NHES mailing. About half 
talked about mail delivery challenges. They did not have extreme opinions or 
experiences for the other factors. 


Sampling Frame Quality 


Three aspects of frame quality were examined as part of the address observations. 


Whether some addresses on the NHES:2019 sampling frame should not have 
been included. There was limited evidence that there are addresses on the sampling 
frame that should not be there. For example, few of the nonrespondent addresses 
could not be located (3 percent). Only 2 percent of the observed addresses were 
determined to be a vacant residential unit or vacant lot, and none were determined 
to be commercial addresses. 


Characteristics of addresses with undeliverable as addressed (UAA) outcomes. 
Addresses that had inconsistent UAA outcomes (that is, some but not all mailings 
returned as UAA) for the NHES screener-phase mailings were more likely to be 
“problematic” addresses. For example, they were more likely than non-UAA 
addresses to be vacant residential units or residential units with unknown occupancy 
status. The few addresses that were observed to be vacant lots all had at least one 
mailing returned as UAA. They also were somewhat more likely to not have a mailbox 
or slot in view. 


Agreement between the NHES:2019 sampling frame and data collected during 
the qualitative nonresponse study. The agreement rate between the sampling 
frame and nonresponse study data ranged considerably across frame variables. For 
data collected as part of the observations, the frame and the observers agreed most 
of the time for occupancy status and structure type, but the agreement rates were 
lower for the presence of children and household income. For data collected as part 
of the interviews, the agreement rates between the frame and interview participants’ 
self-reports were relatively high for Hispanic ethnicity and Spanish-speaking status, 
but they were lower for household income and number of adults in the household. 
This suggests the importance of proceeding with caution when incorporating such 
variables into the NHES design (e.g., for targeting survey materials or contact 
approaches). 
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Chapter 1. Introduction 


Chapter 1. Introduction 


This report presents the methods and findings of a qualitative study of nonrespondent 
addresses to the 2019 administration of the National Household Education Surveys Program 
(NHES:2019). The overarching goal of the qualitative nonresponse study, which included 
both a qualitative interview component and an address observation component, was to 
better understand the drivers of nonresponse to the NHES and to mail-based household 
surveys more broadly. 


This first chapter of the report presents the context for and research questions of the 
qualitative nonresponse study, as well as an overview of the methods used for NHES:2019. 
Chapter 2 provides more information about the qualitative nonresponse study methods. 


1.1 Increasing Survey Nonresponse 


Over the past few decades, surveys have faced persistent declines in response rates (Czajka 
and Beyler 2016; Brick and Williams 2013). The NHES has not escaped this trend. Between 
the first mail-based NHES administration in 2012 and the most recent one in 2019, the 
screener response rate has dropped from 74 percent to 63 percent. In addition, nonresponse 
to the NHES is not random (McPhee et al. 2015; Jackson et al. 2021). NHES nonrespondents 
tend to be younger, less educated, and have lower incomes than respondents. They are also 
less likely to be White or to be married than are respondents, meaning that their exclusion 
from the responding sample can lead to nonresponse bias in survey estimates. 


For mail surveys like the NHES that rely heavily on sample members opening an addressed 
envelope without the encouragement of an interviewer, it is crucial to understand whether 
certain elements of the survey design stop sample members from responding. Having a 
better understanding of these barriers to response is vital for developing NHES survey 
materials or contact efforts to overcome these concerns. In addition, a better understanding 
of the characteristics of nonresponding households could be used to tailor contact efforts, 
assess the accuracy of the information available on the sampling frame, or improve the 
statistical adjustments used to reduce nonresponse bias by increasing the availability of 
measures correlated with both the survey estimate of interest and nonresponse indicators 
(Little and Vartivarian 2005). 


1.2 Theories of Survey Nonresponse 


Given the global nature of the nonresponse phenomenon, researchers have devoted 
considerable attention to understanding the factors that contribute to it. Proposed 
individual-level explanations for nonresponse include privacy concerns, anti-government 
sentiment, busyness and fatigue, concerns about response burden, lack of interest in the 
topic, and low levels of civic engagement or community integration (e.g., Abraham, Maitland, 
and Bianchi 2006; Amaya and Harring 2017; Groves, Cialdini, and Couper 1992; Groves, 
Presser, and Dipko 2004; Singer 2016). Investigators have also suggested that societal-level 
changes, such as the increasing number of survey and solicitation requests, declining 
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confidence in public institutions, and growing concerns about security and identity theft are 
contributing to declining response rates (e.g., Miller et al. 2017; Czajka and Beyler 2016; 
Presser and McCulloch 2011). 


Drivers of nonresponse are complex and interact in countless ways. Much remains unknown 
about sample members’ reasons for nonresponse and the best way to address those 
concerns. Recent task force reports from the American Association for Public Opinion 
(AAPOR) and the American Statistical Association that focus on survey nonresponse argue 
there is a need for more research about the current survey climate and the various factors 
that could improve survey participation (Dutwin et al. 2014; Miller et al. 2017). Questions 
remain about whether research on nonresponse within mail collections from previous 
decades would apply in today’s survey environment. 


The need to better understand the nuances of these drivers and their manifestation in the 
context of a mailed household survey was the motivating factor for conducting this study. 
Below we briefly discuss several of the most relevant theories of survey nonresponse; these 
theories were considered in both the development of the study materials and the analysis of 
the results. 


1.2.1 Person-Level Theories 


These four theories attempt to explain the social psychological influences on sample 
members’ decisions about whether or not to respond to a survey. They focus on the 
individual relationship the sample member has with the survey request, the perceived value 
of the request, and the meaningfulness to the individual or the individual’s self-perception. 


e Social exchange theory: This theory suggests people make decisions about their 
social behavior based on a cost-benefit analysis (Blau 1964). When applied to surveys 
(Dillman, Smyth, and Christian 2014), this theory suggests that sample members are 
more likely to complete a survey request if the perceived rewards of participating 
exceed the perceived costs of doing so. Sample members must also trust that any 
promised rewards will be provided. This theory can be extended to encompass the 
idea that individuals may complete a survey because they have been provided a 
benefit that they are in debt to repay. 


e Cognitive dissonance theory: This theory suggests that people avoid actions that 
result in cognitive dissonance—that is, they avoid doing things that are inconsistent 
with their perception of themselves (Furse and Stewart 1984). When applied to 
surveys, this theory suggests that sample members may decide to respond to a survey 
to remain in line with their perception of themselves as helpful people (Keusch 2015). 
Delaying response (as opposed to refusing) is one method sample members might 
take that allows them to avoid cognitive dissonance without actually responding. 


e Commitment/involvement theory: This theory suggests that commitment and 
involvement are important drivers of a person’s behavior—people are more likely to 
complete/continue with activities when they feel committed or involved (Becker 
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1960). When applied to surveys, this theory suggests that sample members may be 
more likely to complete a survey when they feel committed to or involved with it (for 
example, if they are interested in the topic or care about the sponsor) (Elawad, Agied, 
and Holmes 2016). 


Leverage-saliency theory: This theory suggests that specific survey design features 
have a different amount of leverage on different sample members (Groves, Singer, and 
Corning 2000). In addition, the amount of leverage a design feature has depends on 
how salient it is made during the survey request. Therefore, different sample 
members are likely to have varied reasons for responding, and a given sample 
member is likely to respond differently to multiple survey requests based on which 
design features are highlighted in the requests (Zhang, Lonn, and Teasley 2017; 
Elevelt, Lugtig, and Toepoel 2019). 


1.2.2 Societal-Level Drivers 


Additional drivers may have to do with the societal context in which a survey is conducted. 
These may affect the way sample members interpret and respond to a survey request and, 
ultimately, whether they decide to participate. 


Social integration/isolation: Some researchers have argued that social integration 
has been declining and that this may be a driver in survey nonresponse rates (e.g., 
Abraham et al. 2006). Specifically, as individuals become or feel increasingly 
disconnected from their community (for example, in terms of geography, family, 
social connections, or religion), they may feel less motivation to contribute by 
participating in a survey (Amaya and Harring 2017; Watanabe, Olson, and Falci 
2017). 


Privacy concerns: Other researchers have argued that privacy and confidentiality 
concerns have been growing—and in turn, that trust in the government and other 
organizations has been decreasing (e.g., Singer and Presser 2008; Kim et al. 2015; 
Robertson et al. 2018). This can be a driver of nonresponse, particularly to surveys 
by any entity that is believed to share data or not protect respondents’ privacy. 


Survey fatigue: Others have noted that the number of survey requests made of the 
population has been growing, and they argue this may be a factor in declining 
participation rates (e.g., Presser and McCulloch 2011; Van Mol 2017). The reliance on 
surveys across industries and the ease with which new technologies allow survey 
data to be collected has, perhaps, led to survey oversaturation. 


Being too busy: Finally, it has been posited that individuals are increasingly busy, 
with the pace of life’s demands increasing, and that this “busyness” may stop them 
from responding to surveys (e.g., Ingen, Stoop, and Breedveld 2009; Williams and 
Brick 2018). 
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1.3. Research Questions 


The following research questions guided the qualitative nonresponse study design and 
analysis, with a chapter of this report dedicated to each set of questions. 


e Characteristics of nonrespondents: Who are NHES:2019 nonrespondents? What 
can this study tell us about them that is not available on the sampling frame? What 
are their life experiences, and what do their day-to-day lives look like? What is 
important to them, and what are they concerned about? Can the answers to these 
questions shed light on why they did not respond to the NHES? (Chapter 3) 


e Mail processing: What attitudes and beliefs do NHES:2019 nonrespondents have 
related to receiving mail? How do they receive and sort their mail? Which aspects of 
mailings drive their decisions about what to keep or open? (Chapter 4) 


e Experiences with and attitudes about surveys: What are NHES:2019 
nonrespondents’ experiences with and opinions about surveys? What are their 
opinions about and reactions to the NHES screener mailings and the NHES screener 
questionnaire? (Chapter 5) 


e Typologies of nonresponse: Can we identify typologies of nonresponse among 
NHES:2019 nonrespondents? If so, are they correlated with participant 
characteristics, either based on self-reports or on variables available on the sampling 
frame? (Chapter 6) 


e Quality of NHES sampling frame: How accurate are the variables included on the 
NHES sampling frame? Are there addresses in the sample that appear not to be 
eligible for the NHES? Why do some valid addresses end up with undeliverable as 
addressed (UAA) outcomes? How well do the variables on the frame align with what 
was observed during the study? (Chapters 7) 


1.4 NHES:2019 Methods 


The National Household Education Surveys Program (NHES) was designed to provide 
nationally representative data about topics central to education policy and research. The 
2019 administration of the NHES (NHES:2019) was conducted from January through 
September of 2019. 


Sampling. NHES:2019 used an address-based sample (ABS) covering the 50 states and the 
District of Columbia. The target population was all residential addresses in the United States, 
including P.O. boxes that were flagged by the United States Postal Service (USPS) as the only 
way to get mail. A sample of 205,000 addresses was drawn from a file of residential 
addresses maintained by Marketing Systems Group (MSG), based on the USPS Computerized 
Delivery Sequence File (CDSF). All U.S. civilian, noninstitutional, residential addresses were 
eligible to be sampled. Addresses were selected with differential probabilities of selection 
based on the proportion of households identified as Black and Hispanic in the Census tract 
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in which the address was located. This was done to improve representation of these two 
racial and ethnic groups in the data. 


Data collection. NHES:2019 was administered in two phases: the screener phase and the 
topical phase. 


Screener phase: Data collection began with the mailing of an initial contact letter 
inviting sampled addresses to complete the screener questionnaire. This 
questionnaire asked whether there were any children or youth age 20 or younger 
living in the household (including college students who have no other permanent 
address). The screener questionnaire did not provide any further definition of what 
it means to be “living in the household.” If there were children or youth in the 
household, respondents were asked to provide basic information about each of these 
household members, such as their name or initials, birth month and year, type of 
school enrollment (preschool, public or private school, homeschool, or not enrolled), 
and grade or level of enrollment. See appendix F for a copy of the screener materials. 


Topical phase: Once completed screener questionnaires were mailed back or 
submitted via the online instrument, the demographic information provided in the 
screener was used to determine if there were any eligible children (age 20 or younger 
and with a grade equivalent of 12‘. grade or lower, including children age 6 or 
younger and not yet in school) living in the household—and if so, to sample one of the 
eligible children for a more in-depth topical questionnaire. Parents of sampled 
children were eligible for one of two topical surveys in the second phase of data 
collection: (1) the Early Childhood Program Participation (ECPP) questionnaire for 
children age 6 or younger and not yet enrolled in school (grades K-12); and (2) the 
Parent and Family Involvement in Education (PFI) questionnaire for parents of 
children ages 3-20 enrolled in public or private school in grades kindergarten 
through 12 (or homeschooled for the equivalent). No more than one child per 
household was sampled for a topical survey. Households without eligible children 
were not asked to complete a topical survey. 


The survey included several methodological experiments that varied the offered modes of 
response and the design of the screener mailings (for more information about these 
experiments, see Medway et al. forthcoming). Most of the NHES:2019 sample members 
received a web-push data collection protocol, in which the initial screener mailings offered 
web response and later screener mailing offered paper response, as shown in Exhibit 1.1. 
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Exhibit 1.1. NHES:2019 screener phase contact attempts 


Contact attempt Date Description 

Advance letter! January 8, 2019 This letter introduced the survey, informed the household 
that it had been selected to participate, and provided notice 
of the forthcoming survey. 


Initial screener January 14, 2019 This package included a cover letter, a $5 cash incentive, 
package and a Commonly Asked Questions (CAQ) enclosure. The 
cover letter included the web instrument URL and the 
household’s unique login credentials. 


Pressure-sealed January 22,2019 | This mailing also included the web instrument URL and the 

envelope household’s unique login credentials. 

Second screener February 13, 2019 | The contents of this package were identical to the initial 

package? screener package, with the exclusion of the incentive and a 
slightly different letter. 

Third screener package | March 14, 2019 This package included a cover letter, a paper questionnaire, 
a CAQ enclosure, and a pre-addressed, postage-paid return 
envelope. 

Automated reminder March 14-19, Sampled addresses with a phone number available on the 

call 2019 NHES sampling frame (65 percent of addresses) received 
an automated reminder call. 

Fourth screener April 11, 2019 The contents of this package were identical to the third 

package? screener package, except for the wording of the cover 
letter. 


1As part of a randomly assigned experiment, some sample members were not sent an advance letter, while others were sent two advance mailing postcards 
prior to receiving an advance letter. These postcards were sent in December 2018. 

2As part of a FedEx timing experiment, some sample members were assigned to be sent this mailing via FedEx, while others were sent it via First Class mail. 
Sample members that were sent the second screener package via FedEx were sent the fourth screener package via First Class mail, and sample members 
who were sent the second screener package via First Class mail were sent the fourth screener package via FedEx. 


Most mailings were sent via U.S. Postal Service (USPS) First-Class mail, but one of the 
reminder packages was sent via FedEx (either the second or fourth screener package, as part 
of a FedEx timing experiment).! All materials were developed in both English and Spanish. 
Sample members were sent either English-only mailings or mailings that included both 
English and Spanish materials (i.e., bilingual mailings). Address- and area-level variables on 
the NHES sampling frame were used to determine which addresses should be sent bilingual 
mailings; these criteria differed across the screener mailings, with a larger percentage of 
sample members being sent bilingual mailings as the mailings progressed. 


The qualitative nonresponse study that is the focus of this report drew its sample from 
among nonrespondents to NHES:2019 as of the date that the fourth screener package was 
sent. All the addresses that were sampled for the qualitative nonresponse study came from 
the same web-push condition that was used for most of the NHES:2019 sample members, 
and thus the addresses that were sampled for the qualitative nonresponse study received 
the contact attempts shown in Exhibit 1.1. 


Data collection outcomes. Overall, the screener response rate for NHES:2019 was 63 
percent. The topical response rate was 86 percent for the ECPP survey and 83 percent for 


1 Except for P.O. box addresses which cannot receive FedEx mailings and thus were sent all mailings via USPS Priority mail. 
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the PFI survey. The overall response rate (the product of the screener response rate and the 
topical response rate) was 54 percent for the ECPP survey and 53 percent for the PFI survey. 


For additional details about NHES:2019, see the National Household Education Surveys 
Program of 2019: Data File User’s Manual (Jackson et al. 2021). 
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The overarching goal of the qualitative nonresponse study was to provide actionable 
information about how to address the growing nonresponse problem in the NHES, with the 
goal of increasing the response rate and the representativeness of respondents. 


The study included two components: qualitative interviews and address observations. These 
dual components were designed to provide unique and complementary information that 
together could answer the study research questions and guide the development of new 
nonresponse intervention designs. The objective of the qualitative interviews was to provide 
rich and nuanced information about survey nonresponse. The objective of the address 
observations was to determine the types of addresses that are prone to nonresponse or 
having their NHES mailings be undeliverable and to assess the accuracy of the information 
available on the sampling frame about them. Both components were critical to the study. 


It should be noted that while the study aim was to provide actionable information, this is a 
qualitative research study that was not intended to be nationally representative of NHES 
nonrespondents. It was meant to provide theory building information and should not be 
interpreted as providing nationally representative official estimates. 


The remainder of this chapter provides an overview of the qualitative nonresponse study 
methods, including the sampling, recruitment, and data collection procedures used for both 
components of the study. Additional information about the study methods is available in 
appendix B. 


2.1 Qualitative Interview Methods 


The qualitative interviews took place between May 3 and June 5, 2019. The purpose of the 
interviews was to gain a deeper understanding about how nonrespondents inhabit their 
world and how that orientation may influence their thoughts on research and survey 
participation. The goal was to identify meaningful characteristics of nonrespondents that 
may help generate an understanding of the reasons for nonresponse to the NHES that are 
not already captured by currently available frame variables. 


2.1.1 Sampling 


Sampling for the qualitative interview component of the study was conducted in two phases. 
First, four interview sites were selected. Next, addresses were selected from within each of 
those sites. 
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Site selection 


Because the NHES does not use a clustered sample design, interview data collection was 
restricted to four 30-mile-radius sites to maximize efficiency. The four sites were in 
Connecticut, Ohio, Texas, and California. 2 


To select the four sites, NHES:2016 screener nonrespondent addresses that met a set of 
eligibility requirements were mapped onto a GIS shapefile of U.S. cities and towns. 
Nonresponse that occurred at the screener phase—rather than nonresponse that occurred 
at the topical phase—was the focus of the study because of the impact that screener 
nonresponse has on both the screener response rate and the overall response rate; 
additionally, NHES screener response rates are lower than topical response rates. 
NHES:2016 data was used because NHES:2019 data was not available at the time site 
selection needed to be made, and the 2016 and 2019 sampling and contact methods were 
nearly identical. 


A series of rules were then applied to create an initial list of eligible sites (e.g., there needed 
to be at least a certain number of nonrespondent cases within a 30-mile radius). Drawing on 
the results of a series of algorithms (see appendix B for more details), the final site locations 
were selected to be geographically diverse and to include a demographically diverse set of 
addresses. 


Address selection 


A total of 400 addresses were selected for the qualitive interviews, 100 from each site. 
Addresses needed to meet several criteria to be eligible to be sampled. 


e They did not respond to the NHES:2019 screener request as of the time the fourth 
screener package was sent. 


e They were not classified as a hard refusal (that is, they were not an address that had 
requested not to be contacted again during the NHES:2019 screener phase). 


e They did not have a UAA outcome for any of the first three screener packages. 
e They were located within a 30-mile radius of the site’s center point. 


e They were nota drop point, P.O. box, or rural route address (to facilitate making in- 
person visits to the address). 


2 This represents the state in which the center of each thirty-mile radius site was located. However, some of the sites spanned more than 
one state. 

3 Nonresponse as of the fourth screener package mailing date was used as a proxy for final screener nonresponse because this was the 
best information available at the time sampling was conducted. As a result, some of the sampled addressed ended up responding to the 
NHES after being sampled for the qualitative nonresponse study. Address observations and qualitative interview recruitment were still 
conducted for these addresses. Appendix C includes a sensitivity analysis that was conducted to determine how to treat such addresses 
for the observation analyses; ultimately, it was decided to exclude them from observation analyses. Late respondents were, however, 
included in the interview analyses, and “late respondent” status was used as a subgroup analysis when interpreting the interview 
findings. 
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e Finally, NHES:2019 included several methodological experiments; to be sampled for 
the qualitative nonresponse study, addresses had to have been part of the updated 
web-push mailing protocol. 


Address selection was conducted with the goal of ensuring sufficient representation of 
subgroups of interest among interview participants: (1) households with children, (2) 
Hispanic households, and (3) households with lower levels of educational attainment. Within 
each site, the eligible addresses were divided into eight strata, defined by crossing sampling 
frame indicators for these characteristics. The sampling was then conducted such that, 
across all four sites, a total of 120 Hispanic addresses, 120 addresses with children, and 120 
low-education addresses were selected. 


2.1.2 Recruitment 


Recruitment for the qualitative interviews took place between April and June 2019. A series 
of contact attempts were made to reach each sampled address. Once contact was made and 
the household expressed interest in participating, field staff identified an eligible household 
member to complete the interview. 


Contact attempts 


As shown in exhibit 2.1, recruitment included multiple, varied contact attempts. All 
recruitment materials can be found in appendix G. 


e The study team started by trying to reach sampled addresses by mail. An invitation 
letter and reminder postcard provided an overview of the study and asked household 
members to contact the study team to schedule an interview. The study invitation 
letter also included a $5 cash prepaid incentive, and both mailings noted that sample 
members would receive an additional $120 cash incentive for completing an 
interview. 


e Next, all addresses with telephone numbers available on the sampling frame (about 
270 of the 400 total addresses) received up to two reminder calls that provided an 
overview of the study. If contact was made with the sampled address, staff attempted 
to schedule an interview. If contact was not made, they left a voicemail that 
encouraged the household members to call the study team to schedule an interview. 


e After these initial attempts, field staff conducted a weeklong wave of in-person 
recruitment. At each site, two two-person teams (consisting of an interviewer and a 
supporting field staff person) made in-person visits to the addresses and conducted 
interviews. Each team was assigned half of the 100 addresses that were sampled in 
that site. The teams attempted to visit each address at least during the week. 
Depending on the outcomes of prior contact attempts, some addresses were visited 
more or less often than this. If contact was made with the household, the team 
provided an overview of the study and attempted to schedule an interview with an 
eligible household member. If no one came to the door, they left a Sorry We Missed 
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You card that encouraged the address to contact the study team to schedule an 
interview. 


e Mail, phone, and in-person recruitment continued as shown in exhibit 2.1, with a 
second reminder postcard, a second wave of reminder calls, and two additional weeks 
of in-person recruitment. 


Exhibit 2.1. Qualitative interview recruitment contact attempts 


Contact attempt Date 


Invitation letter April 19, 2019 
Reminder postcard 1 April 26, 2019 

an 1 reminder calls to addresses that had a phone number on the sampling April 26 - May 2, 2019 
Week 1 in-person recruitment May 2 - May 9, 2019 
Reminder postcard 2 May 7, 2019 

Week 2 in-person recruitment May 15 - May 22, 2019 


Wave 2 reminder calls to addresses that had a phone number on the sampling 


frame May 22 - May 29, 2019 


Week 3 in-person recruitment May 29 - June 5, 2019 


Recruitment and interviewing were conducted in both English and Spanish. Addresses 
received either English-only or bilingual contacts using the same criteria as were used for 
the main NHES:2019 data collection. 


Interview participant selection 


Any eligible household member was permitted to participate in the interview. Within- 
household sampling was not conducted. To be eligible, household members had to be at least 
18 and have some responsibility for handling the household’s mail. They also had to have 
lived or stayed at the sampled address for most of the time from January to April 2019. 
Interested household members were screened for eligibility prior to conducting the 
interview. Typically, there was one participant per interview, but in a few cases, a second 
household member also requested to participate or sit in on the interview. 


Recruitment outcomes 


Ultimately, 85 qualitative interviews were completed out of 384 eligible cases. Seventy-one 
were conducted in English only, 9 were in Spanish only, and 5 were in a mix of English and 
Spanish. The response rate was 22 percent, and the refusal rate was 33 percent. Contact was 
made with 64 percent of the addresses.‘ For the refusal rate, addresses were considered to 
have refused if any individual at the address refused; similarly, for the contact rate, 
addresses were considered to have been contacted if contact was made with any individual 


4 Response, refusal, and contact rates were calculated after removing ineligible cases (that is, households that were ineligible to be 
interviewed or were vacant or non-existent units) from the denominator. 
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at the address. More information about the recruitment outcomes, including detailed tables 
presenting the distribution of final case-level outcomes for the in-person and telephone 
recruitment efforts, can be found in appendix E. 


2.1.3 Data Collection 


The interviews generally took place in the participant’s home. In a few cases, at the 
participants’ request, interviews were done in another quiet place. They were conducted 
using a semistructured, qualitative approach and typically took approximately 90 minutes 
to complete. All materials used during the interviews can be found in appendix G. The 
interviews proceeded in several steps. 


e First, the interviewers received informed consent from the participant and requested 
permission to record the interview. 


e They then moved into the semistructured portion of the interview. As shown in 
exhibit 2.2, the interview protocol covered eight domains that facilitated exploration 
into hypothesized drivers of nonresponse and areas of importance to the participants. 
Interviewers were free to address these domains in whatever order felt comfortable, 
conversational, and natural. The protocol included example questions for each 
domain, but there were not any required questions. 


e The interview also included two slightly more structured activities: (1) a mail sorting 
activity with an example mail bundle and (2) an NHES:2019 screener materials 
review activity. Interviewers were again free to incorporate these into the interview 
wherever it felt most natural. These activities are described in greater detail in the 
chapters in which the results are presented (chapters 4 and 5, respectively). 


e Atthe end ofthe interview, participants were asked to complete a demographics form 
that asked about topics such as their age, race/ethnicity, and educational attainment. 
They also were given the $120 incentive that was contingent on completion of the full 
interview. 


Exhibit 2.2. Interview domains and structured activities 


Interview domains and structured activities 
Household make-up Attitudes toward the government 
Experience with mail delivery (includes mail sorting | Education 


activity) Privacy concerns 


Understanding of and attitudes toward surveys Time use 
(includes NHES:2019 materials review activity) Civic and community engagement 


Leading up to and during the interview, when possible, the team also observed unobtrusively 
the home’s exterior and interior to identify information that might provide greater insight 
into the household’s reasons for nonresponse (e.g., what the household appeared to do with 
unopened mail). In addition to the interview recording, the field staff support team member 
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took notes during the interview. After each interview, the interviewer wrote a memo 
summarizing the key discussion points and interview themes. 


Staffing and training 


Given the complexity of the interviewing approach, the interview team was limited to 10 
highly experienced qualitative interviewers. Interviewers completed an intensive two-day, 
in-person training. A field staff person also attended each interview and was responsible for 
taking notes, recording the interview, and assisting with logistical tasks. Field staff 
completed a two -hour virtual training; because most field staff also conducted address 
observations, they completed an additional training as part of that role. 


Short interviews 


After staff had received two soft refusals for the qualitative interview from an address, they 
asked if the household members would instead be willing to complete a shorter interview. 
Six additional interviews of this type were completed, typically at the doorstep of the home. 
After confirming the participant’s eligibility to participate, four additional questions 
assessed whether the participant recalled receiving the NHES mailings, why the participant 
chose not to respond to the NHES, and whether there were any children living in the 
household. Responses to the short interviews are incorporated into chapter 5. 


2.1.4 Coding and Analysis 


After the field period ended, the interview recordings were transcribed. Once the interview 
transcripts were uploaded to NVivo, a series of codes were applied to segments of text in the 
transcripts. More details about the coding process can be found in appendix B, and the full 
codebook is available in appendix G. 


Through detailed reviews of the coded interview transcripts, a team of analysts identified 
key themes in the topics discussed under each of the interview domains. Throughout the 
report, we present participant quotes and anecdotes to support these themes. After each, we 
include the participant’s 4-digit household ID number. Some of the participant quotes have 
been lightly edited for clarity and confidentiality. 


In addition to reporting on overall themes, we also discuss the ways in which experiences 
and opinions differ by key participant characteristics. Throughout the report, we focus on 
the following interview participant characteristics: 


e Characteristics that have been found to be drivers of nonresponse in prior 
NHES administrations: Lower educational attainment, being Black or Hispanic, and 
being Spanish-speaking have all been found to be associated with lower response 
rates to the NHES (McPhee et al. 2015; McPhee et al. 2018). Understanding these 
participants’ reasons for nonresponse is needed to more effectively tailor contact 
protocols and materials that address those concerns. 
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e Characteristics that influence the salience of the survey request: Namely, 
whether or not there were children living in the household. Because the NHES:2019 
screener focused on education and asked about the presence of children in the 
household, households without children may have viewed the survey as not very 
salient and this may have led them to not respond at higher rates than households 
with children. 


e Final NHES:2019 response status: One in six interview participants lived in 
households that responded to the NHES after the fourth screener package (“late 
respondents”). The goal of this analysis was to better understand how late 
respondents differ from final nonrespondents. 


Select analyses include additional interview participant characteristics. These are discussed 
in greater detail where relevant. 


In each section of the report, we discuss only those participant characteristics where 
meaningful findings emerge for that topic. For any characteristics that are not mentioned in 
a particular section, it can be assumed that there were no notable findings related to that 
characteristic for that topic. Given the relatively small number of interview participants and 
the qualitative nature of the data collection, we focus on general patterns for analysis of 
interview data. Statistical testing was not conducted. 


2.2 Address Observation Methods 


Most observations took place between April 29 and May 2, 2019. The objective of the address 
observations was to determine the types of addresses that are prone to nonresponse or 
having their NHES mailings returned as undeliverable and to assess the accuracy of the 
information available on the sampling frame for such addresses. The systematic collection of 
descriptive data identifying characteristics of nonresponding addresses may help generate 
an understanding of the reasons for nonresponse that are not already captured by available 
variables used for sampling and data collection operations. The observations also help 
evaluate the accuracy of frame data. These findings may then be used to identify promising 
options for modifying data collection protocols in future NHES administrations and to guide 
the selection of possible new sample frame sources. 


2.2.1 Sampling 


Sampling was conducted in two phases. First, three additional observation-only sites were 
selected. Then, addresses were selected from within each of those sites. 


Site selection 


Address observations were conducted in seven study sites across the United States. This 
included the four interview sites discussed in section 2.1.1, as well as three observation-only 
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sites in Texas, Illinois, and the Washington DC, area.° These three sites were selected because 
of their proximity to American Institutes for Research (AIR) offices, which increased the 
efficiency of the address observation fieldwork efforts. As with the interview sites, they 
were limited to a 30-mile-radius area to maximize efficiency. 


Address selection 


In total, 760 addresses were included in the observation component of the study: (1) the 400 
addresses that selected for the qualitative interview component and (2) an additional 360 
addresses that were selected from the observation-only sites. 


At the observation-only sites, the address eligibility criteria were the same as for the 
interview sites, with one exception. Addresses were eligible to be sampled if at least one— 
but not all—of the first three screener packages was not returned as undeliverable as 
addressed. The inclusion of these “inconsistent UAA” addresses allowed the address 
observations to gather insight into the drivers of undeliverable outcomes. 


Within each observation-only site, eligible addresses were divided into two strata: 
inconsistent UAAs and non-UAAs. A total of 120 addresses—100 non-UAAs and 20 
inconsistent UAAs—were selected from each site. A systematic random sample was drawn 
from each stratum, after sorting on several indicators that the addresses was in a key 
subgroup of interest and on the 9-digit ZIP code. Thus, although key subgroups were not 
oversampled in the observation-only sites, implicit stratification helped ensure that each 
group’s prevalence in the sample was similar to its prevalence among eligible addresses. 


2.2.2 Data Collection 


Observers drove to each of their assigned addresses. If they could not locate an address, a 
second observation attempt was made later. After arriving at the address, they drove or 
walked around the area to get a sense of the overall neighborhood. When possible, they 
began by discretely taking photos of the exterior of each address, which were used in a 
fieldwork tracking effort. 


Observers next used a tablet to complete a web-based observation instrument. Most 
observations were taken from the street, in front of the sampled address. For multi-unit 
addresses for which the entrance to the sampled unit was inside the building, observers also 
attempted to gain access to the building to observe the entrance to the unit itself and to check 
for the address’s mail access type (i.e., a set of mailboxes in or near the lobby). All 
observations were limited to public spaces, and multi-unit buildings were entered only upon 
receiving permission from building staff. On average, each observation took about 7 to 10 
minutes to complete. 


5 This represents the state or city in which the center of each thirty-mile radius site was located. Some of the sites spanned more than one 
state. 
6 AIR was contracted by NCES to conduct this data collection. 
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Observation instrument 


In the observation instrument, observers began by noting whether they were able to observe 
the address. Next, they completed a series of forced-choice items that asked about the 
characteristics of the address. As shown in exhibit 2.3, the forced-choice items covered two 
types of characteristics: physical characteristics of the address itself and household 
characteristics of the people living at the address. For the physical characteristic items, the 
observer determined which of several available response options best described the address 
(e.g., for structure type, whether the address was a single-family home, a duplex, an 
apartment, etc.). For most household characteristics items, the observer indicated whether 
there was evidence of a particular characteristic (e.g., whether there was evidence of 
children living in the household). For several items, a text box also was included for 
observers to provide more detailed information about what led them to select their response 
to the forced-choice item. The exact number of characteristics collected for each address 
varied depending on other observation outcomes, such as whether the address appeared to 
be occupied (see exhibit B.3 in appendix B). At the end of the observation, observers also 
provided a general description of the address. The full observation instrument is included in 
appendix H. 


Exhibit 2.3. Observation items 


Physical characteristics Household characteristics 

Structure type Presence of children 

Household residential occupancy status Privacy or security concerns 

Mail access type Pride in education 
Patriotism 


Community involvement 
Other household attributes 
Household income 


Staffing and training 


Seventeen staff members conducted observations. Most observers working in qualitative 
interview sites also served as field staff for the interview component of the study. Each was 
assigned 40 to 50 addresses to observe, depending on the site at which they were working. 
Before entering the field, they completed a five-hour virtual training. 


Observation outcomes 


Observations were attempted for all 760 sampled addresses. Of those, 74 percent were able 
to be observed fully. For an additional 17 percent of the addresses, partial observations were 
completed. These were typically multi-unit buildings where the observer could see the 
building but could not gain enough access to observe the entry to the unit itself and thus 
could not determine responses to the household member characteristics items. Finally, 8 
percent of the sampled addresses were not observed at all because they could not be located, 
could not be reached (e.g., within a gated community), or could not be observed for other 
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reasons (e.g., neighborhood safety concerns). For more detail about the observation 
outcomes, see chapter 7. 


2.2.3 Coding and Analysis 


After data collection was completed, a team of coders reviewed the responses that were 
provided in the text fields of the observation instrument to assess (1) whether any of the 
responses should be upcoded into existing categories and (2) whether any new 
characteristic variables should be created. Based on this review, indicators for importance 
of religion, internet or television connectivity, welcoming decor, outdoor living, and other 
outdoor decor were added to the data file. Additional information about the coding process 
is available in appendix B. 


Throughout the report, we also present a series of subgroup analyses that explore whether 
observation outcomes of interest vary for different types of addresses. The variables that 
were used for most these analyses are shown in exhibit 2.4. These variables were included 
on the NHES sampling frame when it was obtained from the vendor, were available in the 
NHES:2019 paradata, or were appended to the sampling frame from other publicly available 
federal government sources. Any exceptions to this approach are noted throughout the 
report where relevant. 


The subgroup analyses generally progressed in two steps. For each outcome, we began by 
conducting a series of bivariate relationships; we used chi-square tests to identify 
statistically significant relationships. For several of the outcomes, we next conducted a 
multivariate logistic regression where the dependent variable was the observation outcome 
and the independent variables were the same variables that had been included in the 
bivariate analyses. We identified statistically significant predictor variables using Wald joint 
significance tests. The goal of this second analysis was to help prioritize independent 
variables. 
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Exhibit 2.4. Variables used for observation subgroup analyses, by source 


Source Independent variable 


NHES:2019 sampling Age of head of household 

frame variable Gender of head of household 
Education of head of household 

Race of head of household 

Household income 

Household flagged as having children 
Number of adults in household 
Phone number available 

Route type 

Dwelling type 

Home tenure 

Urbanicity 

Region 

NHES:2019 paradata NHES:2019 bilingual screener mailings status 
American Community Race/ethnicity stratum 


Survey five-year Tract poverty rate 
estimates (2013-2017) 


Percent of households in Census block that include a child 
Percent of persons in Census block that speak a language other than English 
Percent of persons in Census block without a high school diploma or the 


equivalent 
Decennial Census Low Response Score! 
(2010) 
Federal Residential high-speed internet per 1000 households 
Communications 


Commission tract-level 
estimates (2017) 


1The Low Response Score is a derived variable that identifies block groups with characteristics associated with low mail return rates to the 2010 Decennial 
Census. 
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Chapter 3. Nonrespondent Characteristics and Attitudes 


This chapter describes information collected about NHES nonrespondents beyond what is 
typically available on the NHES sampling frame, such as their attitudes, values, and life 
experiences. Understanding this broader context helps to shed light on their reasons for not 
responding to the NHES. For example, researchers have proposed that declining social 
engagement is a potential explanation for decreasing levels of response to voluntary surveys 
such as the NHES (Abraham et al. 2006). Therefore, collecting information about 
involvement with or connections to communities gives us insight into whether these 
nonrespondents’ experiences and attitudes align with current theories about nonresponse. 


Sections 3.1 and 3.2 focus on findings from the 85 in-depth interviews. The first part of the 
chapter describes the demographics, household composition, and day-to-day lives of 
nonrespondents, while the second part summarizes their attitudes and beliefs. Finally, 
section 3.3 presents address observation findings related to nonrespondent characteristics 
and attitudes. Throughout the report, findings related to the prevalence of the in-depth 
interview themes tend to be presented in terms of proportions (e.g., “about one in three”) as 
opposed to percentages (e.g., “33.3%”). Using this general approach aligns with the 
qualitative, semi-structured nature of the interview data collection and underscores that the 
findings present themes and patterns based on a relatively small number of cases—and are 
not precise measurements that were captured consistently from all interview participants. 
In contrast, findings from the more structured aspects of the study (e.g., address 
observations, interview participant demographics) are reported as percentages because this 
information was collected for all cases. 


3.1 Interview Findings: Characteristics 


In this first section we outline demographic characteristics believed to be drivers of survey 
nonresponse and household composition to better understand the context in which survey 
requests are received. We also explore nonrespondents’ perceptions of how connected they 
feel to the community where they live because feeling disconnected from others and from 
the broader community have been hypothesized to be a driver of survey nonresponse 
(Amaya and Harring 2017). Finally, we examine how they spend their time to better 
understand the competing demands that individuals face for their attention and whether 
general busyness seems to be a driver of survey response decisions. 


3.1.1 Demographics and Household Composition 
Demographics 


As shown in table 3.1, the 85 interview participants reported a wide range of demographic 
characteristics on the demographics form that was administered at the end of the interview 
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(see tables A.3.1 through A.3.3 in appendix A for more information about participant 
characteristics).” 


e They represented a wide range of ages, from teenagers to senior citizens; 28 percent 
were less than 35 years old. Forty-four percent were ages 35 to 54. Twenty-six 
percent were 55 or older. The remainder declined to share their age. 


e Fifty-nine percent of participants were female, and 40 percent were male. The 
remainder declined to share their gender. 


e Forty-five percent had completed high school or less. Twenty-nine percent had 
completed some college but did not have a bachelor’s degree, and 26 percent had a 
bachelor’s degree or more. 


e Thirty-two percent were Black, 29 percent were White, 27 percent were Hispanic, 
and 7 percent reported being part of another racial/ethnic group. The remainder 
declined to share their race/ethnicity. 


e Fifty-three percent reported a household income of $60,000 or less. Thirty-three 
percent reported a household income above $60,000. The remainder declined to 
share their household income. 


e Seventy-nine percent lived in a household where English was the primary language, 
15 percent lived in a household where Spanish was the primary language, and a few 
lived in a household where another language was spoken most often. The remainder 
declined to share this information. 


7 All participant characteristics noted in the bulleted list are based on self-reports provided at the end of the interview. In the few cases 
where more than one household member participated in the interview, the self-reported characteristics are those of the primary 
interview participant. Tables A.3.1 through A.3.3 in appendix A include additional details about participants’ characteristics using 
NHES:2019 paradata and variables available on the NHES sampling frame; appendix B provides additional details about the 
characteristics of NHES:2019 nonrespondents overall. 
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Table 3.1. Number and percentage distribution of self-reported interview 
participant characteristics: 2019 


Number ofinterview Percentage of interview 


Self-reported characteristics participants participants 
Total 85 100.0 
Age 
18-24 10 10.6 
25-34 15 17.7 
35-44 20 21.2 
45-54 20 22.4 
55-64 15 16.5 
65 and older 10 9.4 
Refused $ $ 
Gender 
Male 35 40.0 
Female 50 58.8 
Refused $ $ 
Education 
High school or less 40 44.7 
Some college, but no bachelor's degree 25 29.4 
Bachelor's degree 15 17.7 
Graduate degree 5 8.2 
Race/ethnicity 
White, non-Hispanic 25 29.4 
Black, non-Hispanic 25 31.8 
Hispanic 25 27.1 
Other race, non-Hispanic 5 7.1 
Refused 5 4.7 
Employment status 
Employed for pay 60 68.2 
Not employed for pay 25 31.8 
Enrollment status 
Enrolled 10 11.8 
Not enrolled 75 87.1 
Refused $ $ 
Household income 
$30,000 or less 25 27.1 
$30,001-$60,000 20 25.9 
$60,001-$100,000 15 16.5 
$100,001 or higher 15 16.5 
Refused 10 14.1 


See notes at end of table. 
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Table 3.1. Number and percentage distribution of self-reported interview 
participant characteristics: 2019—Continued 


Number ofinterview Percentage of interview 


Self-reported characteristics participants participants 
Language spoken most often by adults in 
household 
English 65 78.8 
Spanish 15 15.3 
Other $ $ 
Refused 5 4.7 
Child in household 
Yes 40 49.4 
No 45 50.6 
Number of adults in household 
1 adult 25 29.4 
2 adults 40 48.2 
3 or more adults 20 22.4 
Home internet access 
No access + + 
Phone/tablet access only 20 21.2 
Compute 65 75.3 
Refused $ $ 


+Reporting standards not met. There are too few cases for a reliable estimate. 

NOTE: In the small number of cases where more than one household member participated in the interview, the reported characteristics are those reported 
by the primary interview participant. Sample sizes are rounded to the nearest 5. Percentages are rounded to one decimal place but have not been changed 
to reflect sample size rounding. Details may not sum to totals because of rounding. 


SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Surveys Program (NHES), 2019. 


Demographic characteristics that were particularly salient to participants included: 
educational attainment, race/ethnicity, being Spanish-speaking, and socioeconomic status 
(e.g., annual household income and household poverty level). 


Educational attainment. About one in four participants volunteered information about 
their own educational experiences. They attended public and private K-12 schools, trade 
schools, and community and 4-year colleges. In general, participants tended to believe 
education was important, but they had varied experiences with their own schooling. 


Those who held positive views often recalled a supportive teacher, as one participant noted, 
‘I had one teacher specifically, just the way that she taught it, she taught it with passion and 
you could tell that she enjoyed it, and if you had questions, she had no issue answering them for 
you.” (4277) Other participants spoke with personal pride about their education, such as 
being the first to graduate from high school or go to college. One participant relayed a 
conversation that he had with his mother, “You know what?’ I was like, ‘I promise you; I’m 
going to be the first one to graduate.’ And that day when I got my diploma, I felt good.” (6081) 
Others credited their education for their other life achievements: “My career is based on my 
education. You can’t get a job [in my field] without it.” (7303) A few participants recalled 
enjoying their time in school, with one participant commenting that education “helped me 
just reason things out. Just look beyond the surface of things. And dig a little deeper. 
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Consequently, I don’t necessarily believe everything I read. I did like most of the classes that I 
took. I don’t know, give me a broader perspective on things.” (7355) 


Other participants felt negatively about their educational experiences. Some did not feel 
engaged academically, with one stating, “I did great in sports. That was a bigger part of my life 
than academics. I did the academic part well enough so that I could play sports.” (5103) Others 
felt that their education did not prepare them well for the future. As one participant noted, 
“There’s a huge gap in between the level of education you receive at high school, as opposed to 
even just community college. Because I can say from personal experience that I learned way 
more in one semester, in my first semester in college, than I did [in] four years in high school.” 
(6287) 


Some participants’ K-12 educational attainment had been curtailed by extenuating 
circumstances. A few participants had dropped out of school and never obtained a high 
school diploma or equivalent credential. One participant stated, “J dropped out of school in 
high school .... I hated school. It just, it wasn’t challenging enough when I was in school.” (5441) 
Others had to drop out to take care of siblings as one explained, “I didn’t graduate. My mom 
had two more kids. And I raised them. I was [a teenager]. My mom had issues, so I couldn't let 
them be on their own.” (6331) For others, higher education was problematic. Several 
participants discussed its rising costs, which meant that either they, their children, or other 
relatives, did not attend or complete college. For others, going to college had meant taking 
out student loans and a debt burden. One participant related that the school withheld her 
degree until her loan was paid off (7063). Another participant stated, “Oh yeah. I have student 
loan debt. I just basically pay what I have to pay, and, you know, you've got to do what you've 
got to do. And hence why I used to work so much. Yeah. Just to pay bills and not be sick to your 
stomach all the time about where the next money’s coming from.” (7503) While participants 
across all levels of educational attainment shared both positive and negative views about 
their own academic experiences, participants without a college degree tended to express 
more negative views as compared to those with a college degree. 


Race/ethnicity. Approximately one in six participants discussed their experiences related 
to their race or ethnicity. Almost all identified as racial or ethnic minorities. Several 
participants spoke explicitly about immigration. Most participants who mentioned 
immigration were immigrants or children of immigrants. Some of these participants 
expressed concerned related to their legal immigration status, as illustrated by one 
participant saying, “Since we just got our papers about 2 or 3 years ago, I’m afraid they’re going 
to take them away for anything. I take good care of my insurance, because if it is stolen, my 
identity would be stolen; I take good care of what they already gave me because I waited so 
long for it.” (4187) Notably only interview participants who did not end up responding to the 
NHES expressed these concerns. Late NHES respondents who were immigrants did not 
discuss having these kinds of fears. 


About one in seven participants discussed what it was like to have a dual identity (that is, 
living primarily in the United States while having cultural, ethnic, familial, or national ties to 
a different country). They had ties to a wide range of locations, such as Mexico, Nigeria, and 
India. Some of them lived in multi-generational family households (i.e., households with 
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more than two generations). Several shared that household members speak multiple 
languages at home. Of those who discussed having a dual identity, many did not have a 
bachelor’s degree. Several of these participants appreciated that U.S. primary public 
education is free of charge and of higher quality than the education systems in their 
respective home countries. 


Spanish-speaking. As noted earlier, about one in seven participants reported living in 
households where Spanish was spoken most often. Some talked about facing language 
barriers. For instance, one stated that she felt lost when attending events at her child’s school 
because she did not fully understand English (5305). Another participant was used to 
speaking English at work but still preferred reading long letters in Spanish (4017). Another 
participant received help with his college applications from school counselors because his 
parents did not speak English (4655). 


Bilingual family members (usually the youngest generation) seemed to play a crucial role in 
translating for family members who faced language barriers. Some participants stated that 
their children translated for them when needed, while other participants reported 
translating for their parents and other family members on a constant basis. A couple of 
participants mentioned that this was a lot of pressure for them. For example, a participant 
stated that he was the main “filter” between his family and the rest of the world, adding, “J 
am the only one in our family here to go to college, I’m kind of seen as the one that’s kind of got 
to get everything done outside of the household. If we have to go to court for whatever reason 
or something, I got to go because translation and stuff like that. I’m the one to best understand 
what’s going on, [in] situations like that.” (6287) 


Socioeconomic status. Although participants came from a range of economic backgrounds, 
about two in five participants lived in households whose household income was at 200 
percent of the poverty level or below (the cutoff for receiving Medicaid benefits). A few 
participants brought up current or past periods of their life where they encountered financial 
hardship. These experiences included needing to work as children to support their family, 
having to delay college and to worka minimum wage job, having to drop out of private school 
because a parent lost a job, and needing to rebuild financial stability after a divorce. One 
participant described how he and his brother had supported their family financially when 
they were children: 


I’ve been working, gainfully employed since I was 10 years old. My brother and I 
used to deliver newspapers here in [urban city]. We used to get up at 5:00 in the 
morning and deliver the papers, run back home, get an extra hour’s sleep, and 
go to school. Because my parents did not have the resources to give us an 
allowance and even like save for college and everything. That pretty much was 
our deal. (5463) 


Another participant recalled how much the $5 incentive in the study invitation mailing had 
meant to her, saying, “J was broke that day too, so I was like, ‘oh yeah.’ My daughter kept 
complaining, I want some cereal, ‘cause we ain’t have no milk, so, yeah, I bought her some milk.” 
(7275) 
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Household composition 


Participants’ self-reported household size ranged from one to nine people. Just under 30 
percent lived in single-adult households, with about one in five living alone. About half lived 
in two-adult households, and just over 20 percent lived in households with three or more 
adults (see table 3.1 above). Participants’ household members included spouses/partners, 
children (including adult children), parents, in-laws, adult siblings, cousins, friends, or 
roommates. Some participants lived in multi-generational or multi-family households. 


Nearly half of the participants had one or more children age 20 or younger living with them 
(see table 3.1 above).8 The number of children per household ranged from one to five, with 
two children per household on average. About one-third of the participants had school-age 
children or grandchildren (in grades K-12), either living with them or elsewhere. Most of 
these children attended public school; a few attended private or parochial school or 
were homeschooled. Additionally, about one in ten participants lived with children who 
were too young to attend grade school. Of those, about half reported the children were 
enrolled in pre-school. Others noted that their young adult children lived with them some or 
all the time. One participant shared with the interviewer that “I live with my son. He’s a college 
graduate. He graduated last year, and he transitioned back home. And then, my celebration is 
he’s moving out. So, he bought an apartment, and he’s been working, bought his own car, paying 
his own bills. So, he’s transitioning out.” (5195) 


Some participants noted that certain household members only stayed at the sampled 
address part of the time. As a participant said, “My fiancé, he lives in [town], and he comes and 
stays a couple days here. I'll stay a couple days at his house. And I'll go home, and I'll live here. 
It’s working for us.” (5515) Over half of the participants also had family living nearby and 
many stated that relatives or close friends visited or stayed overnight at their home on a 
regular basis. Direct examination of behaviors and perceptions of adults with a tenuous 
attachment to the household is not possible due to the limitations imposed by the participant 
selection criteria. 


3.1.2 Where Nonrespondents Live 


How connected people feel to a given community can influence survey participation. To that 
end, we examined how participants felt about their neighborhood and immediate 
community. 


Why they chose their community 


About one in six participants mentioned how long they had lived in the area, which ranged 
from less than a year to almost 40 years and averaged about 12 years. A few participants had 
been in the community for their entire lives, such as one participant who lived in a home that 


8 This is based on self-reports provided at the end of the interview. Because the target age for the PFI topical survey ranges up to age 20, 
the demographics form that interviewers administered asked if there was anyone age 20 or younger living in the household. For ease of 
discussion throughout the report, we refer to these household members as “children” even though some of them may have been over 18 
years old. 
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her family had owned for over a century (5479). One in four participants described why they 
chose their community. Reasons included (from most to least commonly mentioned): 
proximity to family and other loved ones, access to employment opportunities or proximity 
to work, safety, and the quality of local schools. Participants often had more than one reason 
for living in a particular area—for instance, some participants who moved to an area for 
access to employment also had family in the area. Yet others chose their neighborhood 
because it was both close to their job and felt safe. 


Most participants who chose their community due to proximity to others had moved there 
to be close to family members. For instance, one participant explained that she chose her 
current neighborhood because it was close to her sisters: “I have a sister who lives right down 
the street in that house, so not far away. And then I have another sister that lives up this way in 
another set of apartments.” (4495) Another participant rented the apartment right above her 
Spanish-speaking parents so her son could be exposed to Spanish (5195). 


Employment opportunities were another leading reason why participants chose to live 
where they lived. For instance, one participant had moved to a new state because there were 
better opportunities for teachers (4167). As the participant explained, “... my husband and I 
moved out here [from another state] mainly because my husband's a teacher and teachers [in 
that state] are paid pitiful, or at least were then. And his sister and her husband ... were urging 
us to come out because the need for teachers [here] was great and they pay more.” 


Safety was another important factor in participants’ choice of where to live. These 
participants tended to mention characteristics such as safe, quiet, calm, and secure when 
describing their neighborhood. As one participant recounted, “we chose [this house] because 
the neighborhood [is] very quiet and secure, and very low in crime; we love it here.” (4701) A 
few participants had moved to their current neighborhood because their previous 
neighborhood was unsafe. For instance, one participant noted that her current 
neighborhood was “very peaceful,” compared to her previous neighborhood which was “not 
very safe.” (4187) 


Finally, the quality of local schools played a role in participants’ choice of neighborhood. 
Several participants either moved to the area in which they lived specifically because of the 
quality of its school system or planned to leave their current location because of its poor 
public schools. For example, one participant explained that she picked her neighborhood 
because the “best elementary school” was “down the street,” (4025) while another indicated 
that she had moved to the area because it had a “better school system.” (4129) Conversely, 
one participant noted that she had considered moving to have access to a better public school 
system: “[Suburban town’s] education system, I don’t know, it’s kind of iffy. If | had the means 
to move I probably would, but it’s not horrible.” (7303) A few participants noted that their 
neighborhood was difficult to get into because of the quality of the school district. 


Satisfaction with where they live 


Participants generally described being happy where they lived. Safety was the primary 
driver of whether participants were happy with where they were living. Many of those who 
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mentioned that they liked their neighborhood/community did so because it was a safe place 
to live. In particular, participants who had children in their households mentioned wanting 
to live in places that were quiet, secure, and safe for children to play outside. One participant 
explained that she liked living in her community because “children, sometimes, they're playing 
outside and there’s no problem with that.” (4541) Another participant echoed this sentiment, 


saying that he sought to live in a neighborhood where “you can be relaxed when kids are 
playing outside.” (4705) 


Participants’ experiences of lack of safety varied greatly. While one participant’s primary 
safety concern was the fact that it was not safe to leave packages on the porch in her 
neighborhood (7553), another described a harrowing experience of hearing shots fired near 
a family cookout. As the participant explained: 


My nieces and nephews [were] running around. All of a sudden, a car just drives 
by, pop, pop, pop. That made the whole cookout just want to just bring the kids 
inside because it wasn't safe. It’s just been like that for the past couple of years. 
It’s more violence than the community actually helping each other. (5757) 


A few participants stated that they wanted to move out of their current 
neighborhood/community because of safety concerns, with one stating: “This area, I won’t 
lie to you, I want to move. ... When you park in some gas station, you see people bringing stuff, 
selling [drugs] here. In this area, it’s so rough.” (4705) 


For some participants, looking out for the safety of their neighbors helped them feel 
connected (see “Sense of Belonging” in section 3.2.1 for more details). As one participant 
described: 


We know, we look out for each other. ... People go through what they have to do. 
If we see each other, you wave and you'll sometimes talk. ... So everybody just 
gets along. They go [abroad on vacation], they'll ask me to just keep an eye on 
their house. And we believe in being peacemakers. Your neighbors are who you 
can see and talk to. ... It’s like, eyes, everybody look out for one another, eyes on 
eyes. (5409) 


Experiences tended to vary across different subgroups. Hispanic and Black participants 
reported being unhappy where they lived more often than did White participants and were 
particularly concerned with safety in their neighborhoods. Further, participants who did not 
have bachelor’s degrees seemed to be more concerned with safety than those whose highest 
level of education was a bachelor’s degree or more. 


3.1.3. How Nonrespondents Spend Their Time 


Understanding how individuals spend their time helps us better understand the competing 
demands on their time and the day-to-day context in which survey requests are received. 
Participants shared their experiences with and perceptions about paid work, going to school, 
household and caregiving responsibilities, civic and community involvement, recreation 
activities, and internet and social media use. 
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Paid work 


Sixty-eight percent of the participants reported that they were currently employed for pay 
(see table 3.1 above). Participants from households with children reported not being 
employed for pay more often than those without children, and White participants reported 
not being employed for pay less often than Hispanic and Black participants. Those without 
bachelor’s degrees also reported not being employed for pay more often than participants 
with bachelor’s and graduate degrees. About one-fifth of participants under 65 years of age 
were not working; most of these participants were women of color. They were not working 
either by choice (e.g., as a stay-at-home parent) or because they had not been able to secure 
steady employment. 


Type of work. Participants reported a variety of paid work activities. Job types tended to fall 
into six groups (in order from most common to least common): 


1. Manual labor: Work in construction, home repair, sanitation, or transportation 
2. Health care: Work in hospitals, clinics, or group homes 

3. Service industry: Work in restaurants, stores, or salons 

4. Executives: High-level, typically managerial, jobs 

5. Education: Teachers in K-12 schools or in universities 


6. Miscellaneous: Participants in this group held a variety of jobs not prevalent enough 
to have their own group and included a newspaper editor, a jewelry maker, and a 
restaurant owner. 


A handful of participants reported working multiple jobs and several worked one or more 
jobs while they attended school. 


Satisfaction with work. Some participants commented on their job satisfaction. Several 
participants—especially those in executive and health care positions—reported enjoying 
their jobs and being excited to go to work. For one participant, her job as a medical records 
clerk gave her satisfaction because she helped people during a difficult part of life. “J love my 
job. ... I enjoy the satisfaction of taking care of people and making a difference at the end of 
their life, if 1 can make them happy in any way while they’re dying.” (7503) One participant 
who was an accountant explained that his job brought him enjoyment in addition to paying 
his bills: “I’m making a living. I’m paying my bills, but I’m enjoying what I’m doing.” (5463) 
Another participant who worked as a consultant for lighting design stated that he loved his 
job because of the independence (7627). 


A few participants stated that they were not happy with their jobs in some way. Most worked 
in manual labor or the gig economy. Some participants had stressful jobs or worked long 
hours that made them feel burned out. One participant who worked the nightshift at a 
newspaper stated, “My current newspaper is, because of what’s going on these days, just dying. 
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I’m just working so many extra hours. They've given me more duties I really don’t want. Come 
weekends I really don’t feel like going out to parties. It just exhausts me.” (7149) Other 
participants were not happy with other aspects of their jobs, such as the travel requirements 
or commuting time. For instance, one participant who worked as a correctional officer and 
spent several days away from home at a time, also wanted to find a job that would allow him 
to be closer to his family: “Once I get the job, I’m going to quit the prison. At least, going [back] 
and forth. I want to be in one place. So that I can be with my family every day, and go to work 
and come back, come down and stay alone. Because I get tired of staying alone.” (4705) 


Erratic work schedules. Across all job types, many people worked long hours. One in five 
participants reported having erratic schedules, either at work or while juggling work and 
other responsibilities, such as school or family. 


In some cases, participants had work schedules that were unpredictable or required travel. 
For instance, one participant who worked for a clothing company stated that her work 
schedule changed constantly and made planning for social activities difficult: “The schedule 
changes and everything changes. ... Sometimes, weekdays I never go out. Almost never.” (5429) 
Another participant who worked for a home-improvement company stated that his work 
hours vary (6221), while a participant who was a nurse in an emergency room had a work 
schedule that changed according to the needs of her department (7015). Overall, those who 
did manual labor were particularly likely to report having unpredictable work schedules. 


A few participants had erratic schedules due to holding multiple jobs. These participants 
tended to work in the health care system or in manual labor. Some worked different jobs on 
different days of the week, and others worked different shifts for different jobs. A few 
participants who were students also had erratic schedules. In one case, a participant worked 
multiple part-time jobs and also was attending school (6287). In others, participants juggled 
child care and paid employment while attending school. 


Hispanic participants reported having erratic schedules due to multiple jobs more often than 
White or Black participants. Participants from households with children reported having 
erratic schedules more often than those without children. In addition, participants from 
households with children reported having multiple jobs and juggling work and other 
responsibilities more often than those without children, though only one directly mentioned 
child care responsibilities as a reason for having an erratic schedule (5691). This participant 
noted that her weekdays were “crazy,” and she took her daughter along with her on errands. 


Participants across educational backgrounds reported having erratic schedules with similar 
frequency, though the reasons for the erratic schedules tended to be different. Participants 
with bachelor’s degrees and higher tended to report erratic schedules due to work travel and 
fluctuating work schedules, whereas participants without bachelor’s degrees reported 
having erratic schedules due to holding multiple jobs and long workdays. Lastly, late NHES 
respondents reported having erratic schedules more often than final nonrespondents did. 
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Educational enrollment 


Twelve percent of participants reported that they were enrolled in school at the time of the 
interview (see table 3.1 above). These participants were pursuing a variety of credentials, 
including GEDs, associate’s degrees, bachelor’s degrees, doctoral degrees, and professional 
certification programs. Most of these participants were also parents of school-age children 
and mentioned having to fit their studies in with their children’s school schedule. One 
participant described her routine in the following way: “I do go to school, full time, at [local 
community college]. I go there from about 9:00 in the morning till around 2:30 or 3:00, every 
day in there. [I] wake my son up, and we take the babysitter, and we’re pretty much here until 
we start over again the next day.” (6587) Almost all these participants were working in 
addition to going to school. As one participant summarized, “Yeah, I work full time and go to 
school full time. So I’m, like, always tired.” (4639) 


Household responsibilities, caregiving, and involvement in children’s education 


Most participants regularly ran errands and spent time every day on home upkeep, such as 
cleaning, laundry, and food preparation. Some participants stated that they disliked the daily 
grind of running errands and doing household chores. For instance, one participant 
explained: 


I hate grocery shopping and cooking dinner. ... Yeah, I think it’s a big time-sucker 
of mine. I’m either making breakfast or I’m packing lunches or I’m making dinner 
or I’m shopping for food that they eat in two days. Oh, I’m at the grocery store at 
least three or four times, but I do one day of big grocery shopping and it doesn’t 
matter. Yeah, it’s a big time-suck. (5203) 


When not working for pay, many participants were caregivers for children. Reported 
caregiving activities included supervising small children, transporting children to and from 
activities, and helping children with school assignments. Most participants who described 
caregiving activities were women. For instance, one mother described her daily caregiving 
responsibilities in the following way: 


Well, I work in the mornings. In the afternoons I pick up my son and nephew 
from school, and I pick up my son with my nephew. We stay here at home. 
Sometimes we go out to get some lunch, and we come back and stay here 
together. At night, I give them a shower, I get them ready and we go to sleep. 
And again the next day. (4541) 


For another mother, the weekday routine of household and caregiving responsibilities was 
exhausting: 


Because Monday through Friday is like a whole routine every day. It’s like come 
home, cook, kids. The day is not enough. I have to get up extra early, because I 
know I have to take my daughter to school, no matter what, be at work on time 
... get out of there ... yes, no matter what. Get out of there on time, come and get 
my daughter. ... By the time I come home, it’s already 6:00. I still have to do 
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dinner, because this one wants to eat. ... oh, my God, I get so overwhelmed. ... Next 
thing you know it’s already 8:30, 9:00 and I can go to bed already, and start all 
over the [next] day again. (6201) 


Caring for their grandchildren was also mentioned by several women. For instance, one 
grandmother reported taking care of her grandchildren everyday while her daughters went 
to work (5693). This participant noted that she did not have time for volunteering because 
she spent so much time caring for her grandchildren. 


A few participants also described caring for spouses, and other family members—including 
taking care of ill family members. For example, one participant mentioned that her day 
revolved around taking care of her husband's diabetes: “Well, I get up in the morning, I take 
care of everything, I have an hour to myself. My husband, he’s a diabetic, so I make sure I have 
everything going for him. That’s the whole day.” (7597) Another participant had spent several 
years caring for her daughter, who had a terminal illness (4043). 


Parents of school-age children described ways that they supported their children’s 
education, including overseeing homework, transporting them to afterschool activities, 
participating in educational experiences with children outside of the classroom, and 
communicating frequently with teachers and school leadership.In addition, some 
participants who had grandchildren or other young relatives who lived nearby also actively 
participated in their school-based activities. Many participants noted that supporting 
children’s education was very time consuming and, at times, overwhelming. For instance, 
one participant reflected on parents’ involvement in education in the following way: 


So, it’s tough, because again this has changed a lot since we were kids. It was 
more passive back then. Parents weren't as involved as they feel like they need to 
be today. Speaking from experience, like I said, my older son, he needs a lot of 
assistance. I’m constantly in the school, doing 504’s [plans developed to ensure a 
child who has a disability and is attending an elementary or secondary 
educational institution receives accommodations] and __ re-writing 
accommodations and doing all of that. (5103) 


Managing out of school educational activities in addition to helping with schoolwork 
compounded time pressures. One participant described taking her children to a series of 
afterschool activities, which took a substantial amount of time: 


They go to afterschool programs, just like extensions of different enrichment 
programs. Like band school and then my older one is part of a volunteer program 
at the library. He does that a couple times a week. They also do some afterschool 
programs at the library as well. ... So, I have to pick him up and then go to 
another school to pick them up and then disperse them after school to all the 
little places and then pick them all back up. Yeah, it’s a lot. (5103) 


How often parents did these activities differed. Some had work or other commitments that 
led them to participate less frequently than they would like. One participant noted: “J 
remember once [parents] had to participate, to be there in class with the children ... but there 
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was no way to attend because of work. ... Not because I didn’t want to do it, but for not 
neglecting work responsibilities.” (4131) This participant wished that the school had more 
activities in the evenings, so that working parents would be able to participate. 


Further, a few monolingual Spanish-speaking parents noted that they had difficulty 
communicating with their children’s schools because of language. As an example, one 
participant stated “I don’t like ... that they don’t speak very much Spanish [at school]. 
Everything is in English, everything is in English. So, at PTO, I go, but I’m kind of lost.” (5305) 
However, one participant explained that she does get support, in Spanish, from her 
daughter’s teachers: “The teachers call to give us reports or they call to give us some advice 
about how she’s progressing and how to get [her] to practice more with what [she is] behind 
in.” (4017) 


Busyness 


More than half of the participants talked specifically about being extremely busy. Even those 
who did not explicitly say they were extremely busy often had many activities or 
responsibilities in their lives. Several participants used the weekend or downtime to catch 
up on sleep. 


Work responsibilities dominated the time of many participants. One participant reported 
working 10 to 18 hours a day during the week at a cafeteria and working at his own food 
truck on weekends (6681). Participants who reported long work hours had a wide range of 
occupations, including an accountant who worked around 60 hours a week (5463), a clothing 
store employee who worked around 10 hours a day (5429), and a correctional officer who 
worked 12- to 16-hour shifts 4 days a week (4705). Feelings of busyness were compounded 
for some because household members worked different schedules. This was illustrated by 
one participant who attended a local community college 9:00 am-2:30 pm and cared for her 
child in the afternoon, while her husband had two jobs with a rotating schedule, working 
2:00-10:00 pm on some days and 10:00 pm-6:00 am on others (6587). Similarly, in some 
households, multiple adults held more than one job. 


As noted above in the caregiving discussion, family commitments, particularly with young 
children or grandchildren, were extremely time consuming. One participant remarked on 
how busy life was for her as a stay-at-home mother of four: “So I didn’t know how many 
errands need to be done throughout the day ... [The kids] have to go to school, and then they 
have sports, and then we have to go to the library and then they have to go to the science thing 
because they've got to get this project done.” (6793) When asked what kept her from filling 
out the NHES, another participant referred to her children by saying, “I look at something, 
and then somebody needs something and a bunch of other things and [you] forget what you 
were doing in the first place.” (7303) 


Other households had a mix of work and family responsibilities. For example, one participant 


who was a mother and worked the night shift at a hospital described how busy she was with 
work and family responsibilities. She described her typical day as follows, “Well, getting off 
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of work, coming home, washing clothes... cooking, and going to sleep. Literally my life is taking 
care of [the kids] and going to sleep. Work and sleep, that’s it.” (7057) 


Participants from households with children, in particular, reported being extremely busy. 
They tended to say that they were so busy with their daily routines—including working, 
caregiving, supporting children’s education, and doing household chores—that they did not 
have time for socializing or self-care, much less filling out surveys. 


Though busyness was reported by both late respondents to the NHES and final 
nonrespondents, it was mentioned more often by late respondents. Participants whose 
highest level of education was a bachelor’s degree or more reported being busy more often 
than those who did not have a bachelor’s degree. Participants with a bachelor’s degree or 
more reported being busy primarily due to long work hours and juggling work and other 
responsibilities, such as family. Those with some college or less reported a wider array of 
reasons for being busy, including working multiple jobs. 


Recreation activities 


Despite being busy, participants also described a variety of activities in which they were 
engaged when not working or taking care of the household or other household members. 
These activities tended to fall into seven groups (from most to least common): 


e Sports/exercise: Going to the gym or participating in group sports, watching 
children or grandchildren play sports, or watching professional or college sports 


e Outdoor activities: Hiking, long walks, camping, or visiting parks 


e Event-based outings: Eating out at a restaurant, visiting the zoo or an amusement 
park, going to the beach 


e Religious activities: Attending worship services or Bible studies, praying, 
meditating 


e Entertainment: Watching movies or television, reading, dancing, online gaming, 
playing board or card games 


e Volunteering: Running a food pantry, assisting ata women’s crisis shelter, beach 
clean-up 


e Arts-based activities: Singing, writing, playing instruments 


Most participants talked about fun and relaxing activities as ways of connecting with others. 
For example, one participant whose daughter had recently died noted that she went fishing 
every day with her friend, brother, and sister to “get more involved with life again.” (4043) 
Another participant did country western dancing to unwind and stay connected with others: 
“So, country western dancing, I’ve been doing it for 9 years now. And it’s just an escape. Just go 
out, hang out with friends and dance, and not really worry about anything.” (4209) Some 
participants had the opposite approach—they desired solitude when wanting to have fun or 
relax. One participant noted that she was a “homebody” and rarely went out on the weekends 
(4495). 
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Some participants noted that they did not have time to engage in relaxing or fun pursuits 
because of work or family demands. One participant, for instance, noted that she did not have 
much time for relaxing activities since being promoted (4129). Another participant, who was 
a mother of three, noted it was “very hard to schedule time away” for fun activities with her 
husband; “Maybe once a year we'll get out and do something.” (5103) Other participants 
explained that they made a point to do things as a family, particularly on weekends. Likewise, 
another shared that on weekends she did “everything as a family,” including errands, her 
children’s sports events, picnics, eating out, and trips to the beach (5691). 


Internet and social media use 


Three-quarters of the participants reported having home internet access via a computer (see 
table 3.1 above); most of them also had access via a smartphone or tablet. Just over 20 
percent only had home internet access via a smartphone phone or tablet. A few did not have 
internet access at home at all. 


Participants mentioned using the Internet in five primary ways (in order from most to least 
commonly mentioned): 


e Banking and bills: Using online banking and bill pay 

e News: Watching or reading news via online news sources or aggregators 

e Entertainment: Watching streaming video services, and gaming with others 
e Socializing: Using social media platforms 


e Shopping: Using online shopping sites to make purchases 


Participants considered the Internet to generally be untrustworthy, either because of news 
reports or negative personal experiences. The extent to which they distrusted the Internet 
varied, from those who were suspicious of certain information requests online (such as 
requests for survey participation) to those who distrusted it more generally. Several 
mentioned being the targets of fraud or scam attempts, which made them suspicious of 
online security (see section 3.2.3 for more details about privacy concerns). Several 
participants mentioned social media when discussing the Internet, but only a few did so in 
conversations about activities they found fun or relaxing. For instance, one participant 
mentioned that she had deleted her Facebook account because her friends would not react 
positively to some of her posts (7063). Another participant added that “you can’t always trust 
what’s on Facebook, or Twitter. You never know if those are real sites, or, you know.” (4277) 


3.2 Interview Findings: Attitudes 


This second part of the chapter focuses on attitudes and beliefs that are hypothesized to be 
particularly salient to survey participation decisions. We explored community and civic 
engagement and privacy concerns because the dissolution of the first and the rise of the 
second have been cited as drivers of survey nonresponse (Amaya and Harring 2017; 
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Robertson et al. 2018). We also explored attitudes about education and opinions about the 
government because attitudes toward the survey topic and sponsor have also been cited as 
drivers of survey nonresponse (Groves et al. 1992; Groves et al. 2014). 


3.2.1 Community and Civic Engagement 


Because lack of community or civic engagement has been hypothesized to be a driver of 
survey nonresponse (Abraham et al. 2006), interviewers asked participants about a variety 
of topics related to these types of engagements. 


Community 


About one in five participants spoke explicitly about what community meant to them. This 
coalesced around two themes—getting together with other people and helping/caring for 
neighbors in some way. 


Getting together with other people. For some participants, particularly those households 
with children, community meant getting together with other people in the neighborhood 
around shared activities or interests. These participants described attending informal social 
events (e.g., barbecues or pick-up games) and structured activities (e.g., local sports 
programs, religious services, or support groups). For example, for one participant, 
community meant gathering for spontaneous social activities, whether in person or online. 
She stated: 


When I think ‘community’, I think everyone from our apartment complex 
meeting at the pool and having a barbecue and a couple of drinks, and playing 
some volleyball, water volleyball. Just a big group of people getting along, 
essentially, to boil it down to the most basic. ... A group of people all getting along 
and building each other up is community. Whether that be across the Internet, 
or it be next door neighbors. (4209) 


Some participants who took part in structured community activities tended to see these as 
support networks. For instance, a participant volunteered that her church’s outreach 
activities helped with local community building: “So, we do a lot of community things. We feed 
the hungry. We give them care packages, give them the word of God and give them hope.” 
(5409) Another participant described his participation in Alcoholic Anonymous (AA) as a 
way to support the community: “So, I was ... involved in a small population of the community. 
... 50, | had quit drinking and Iam a member of AA. So, I was actually secretary of a group and 
stuff like that. Trying to help out individuals.” (7329) 


Helping and caring for neighbors. For other participants community meant helping or 
caring for their neighbors in some way. This understanding of community was often 
mentioned by final NHES nonrespondents and those with a bachelor’s degree or higher. 
These participants tended to describe knowing their neighbors well and having caring 
interactions with them, such as watching out for children as they played, giving assistance to 
neighbors, and communicating about crimes that happened in the neighborhood. For 
instance, one participant stated that he helped neighbors with car repairs for free: “I work on 
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people’s car without pay. Free. You can go outside and ask most of the people here. They'll say, 
oh, if I have time, I let them know. ... Mostly ladies. Because there are some, their husbands are 
not there. I feel for them.” (4705) Another participant noted that she felt a sense of community 
because her neighbors always communicated about crimes or safety issues in the 
neighborhood. She described: “It’s more like communication. ... Like seeing weird things and 
communicating to each other. ... I think a neighbor’s son was mugged but far away. ... And the 
next day, they all talked ... that [if] you are walking there, by your car, well, be careful or things 
like that.” (4541) 


Some participants noted that connection and trust were important dimensions of 
community. These participants tended to value knowing their neighbors and being well 
known in the community. One participant stated, “Community for me is like recognition and 
connection and trust. And in the neighborhoods, we know each other.” (5195) The same 
participant then described the type of neighborly interactions that supported this feeling of 
trust: “Like the neighbor across the way, he’s an [elderly] very grumpy gruff. And yet, every 
single time he sees us, [he asks,] “How’s your son doing?” 


Sense of belonging 


About three-fifths of participants had a sense of belonging or connection to their 
neighborhood or community. Participants with a bachelor’s degree or higher, White 
participants, and late NHES respondents reported feeling connected to their neighborhood 
more often than other groups. The reasons participants felt connected varied (in order from 
most to least commonly mentioned): knowing or interacting with people in their community, 
being involved with their children’s school, engaging in public service through work or 
volunteering, and being connected to an immigrant or ethnic community. 


Knowing and interacting with neighbors. Several participants mentioned that they felt a 
sense of belonging through knowing people in their community and having neighborly 
interactions. This was often noted by final nonrespondents to the NHES, participants from 
households with children, and participants who did not have a bachelor’s degree. These 
participants noted a range of interactions with neighbors, including greeting, having 
conversations with, visiting, and giving assistance to neighbors when needed. For instance, 
one participant stated that he felt a part of his community because of tight-knit relations with 
his immediate neighbors: 


I guess, based on my history of the neighborhood, I feel a part of the community. 
I know [participant’s neighbor] across the street. My mom’s cousin lives in this 
apartment right here. We’re friendly with the neighbors that live next door. ... On 
this street, it’s a very tight-knit community I would say. Everybody knows 
everybody, but once you start to venture off the street, then it’s a typical 
neighborhood. (5479) 


For another participant, whose interview took place at a nearby coffee shop, knowing people 
there made him feel like he belonged to the community: “Most of the time I come in here, 
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everybody knows me and stuff. So yeah, I’ve been coming here for a long time. ... So that’s kind 
of like my community I think.” (5685) 


Being involved with school activities. Another set of participants felt a sense of belonging 
through participating in their children’s school-based activities. This was often mentioned 
by participants with a bachelor’s degree or higher. These participants tended to get to know 
other parents and school staff who were also involved with these activities. For instance, one 
participant described feeling part of a “subcommunity” of sports programs in her town: 


I feel like a part of our community, because like I said, we’re very integrated into 
the whole sports programs here and [at] school and you get to know all the 
parents, teachers, and leaders and everybody else that lives around [here]. You 
do become part of that as well. Although, they’re smaller groups within a larger 
community. ... Within our big [town] community, there are subcommunities that 
I definitely see. Because they’re all in school and sports and that’s where we are 
right now. (5103) 


Another participant described making friends through his daughter’s preschool: “At 
preschool, there’s a lot of parent involvement. You get to be in the classroom and help with daily 
activities. It’s really fun. Help plan all the big events at the school. ... Since I’m not from here, 
I’ve made a lot of friends, which is really nice.” (7303) 


Public service through work or volunteering. Some participants mentioned that they felt 
a sense of belonging to the community through engaging in public service. The reported 
volunteer activities involved a range of organizations, such as nonprofits, religious 
organizations, sports organizations, school-related organizations, and fundraising efforts. 
One participant was active in volunteering opportunities that she found through her job: 


I like to help ... Now with my new profession, I like to go and help with more 
functions that are through our work. So, we do [sporting events], my daughters 
and I would go volunteer to work the [sporting events]. Sometimes first aid 
booths at festivals [...] like I said right now, a lot of the events that I volunteer at 
with my daughter, are through my work. They'll notify us when there are things 
happening that we might need some help. (7015) 


Another participant described feeling a sense of belonging to the local community through 
her involvement with various groups: “I belong to [local nonprofit organization] here in 
[urban city]. It’s a group that ... [tries] to help people who are in poverty or who need a nudge 
... either socially or economically or [with] friendship or stuff like that.” (5463) 


Being connected to an immigrant or ethnic community. For Hispanic participants and 
participants from Spanish-speaking households, this feeling of belonging seemed to stem 
from being connected to ethnic/immigrant communities or, in one case, from having 
obtained U.S. citizenship. One participant noted that she returned to her neighborhood in 
part for the sense of community she found through the Puerto Rican community (5195). For 
some Hispanic participants, obtaining legal status was an important factor in feeling like they 
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belonged to their community. One participant explained that she had felt “more part of the 
community” since obtaining legal status in the United States (4187). 


Lack of belonging or connection 


In contrast, some participants described feeling disconnected from people, groups, or their 
local community. Some participants noted that they were new to the area. One participant 
shared, “Just moved in here. I’m pretty friendly with the neighbors next door. They come out to 
say hi or whatever. But other than that, no, I’m not really connected with the neighborhood.” 
(7057). 


Other participants noted that they kept to themselves on purpose, for reasons such being 
introverted or not feeling like they fit in. For instance, one participant noted, “I kind of keep 
to myself. I talk to people on the third floor, that’s just because I’m related to them. ... It’s not 
that I don’t like people, it’s I have, like, severe anxiety around people.” (5449) Another 
participant stated that she had never fit in to the places where she lived: “Honestly, I’ve always 
kind of felt like the odd one out anywhere I go. Because I just don’t follow the crowd. Everyone 
around me could be looking to the left, I'll purposely be staring to the right, just because I don’t 
like to ... I like to dare to be different.” (5265) 


A few participants mentioned that they did not feel connected to their neighbors because 
they did not like them, with one participant saying, “Other than working there I don’t really 
know my neighbors. I don’t really like my neighbors. It’s like older people. They have a lot of 
dogs barking.” (5441) Some participants from immigrant backgrounds mentioned that they 
were disconnected from their surrounding immigrant or ethnic communities. One 
participant from a Latino background, explained that the other Latino-Americans who he 
knew had taken a different path in life than he had so he did not feel very connected to them 
anymore: 


There’s two different cultures that have been produced from the immigrant 
parents, right? Because some go off to college, and become successful and stuff 
like that, which is where I want to go. And then you have other cousins who came 
from the same, literally cousins I grew up with, ... the same situation as me, but 
they chose a different route. Where it’s like, “I don’t want to go to school.” And 
they get involved in gangs and stuff like that, and drugs, and they are in and out 
of jail, and stuff like that. ... Well, [now] I don’t really see anybody else outside of 
my family who’s [Latino] ... In high school, I guess you could say that it was like 
... because it’s your same age group, and especially in this area where most 
people are [Latino] ... we can relate to them a lot more. But yeah, but even them, 
it’s like the same thing as with my cousins, they [took] different routes in life. So, 
it’s just life, I guess, because I don’t really feel too connected to them anymore at 
least. (6287) 


Feeling a lack of belonging to the community varied by participant characteristics. For 


example, final NHES nonrespondents reported not feeling a sense of belonging more often 
than late respondents. Final nonrespondents who felt a lack of belonging tended to be new 
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to the area, whereas late respondents who expressed this feeling tended to report keeping 
to themselves on purpose. In addition, participants from households with children reported 
a lack of belonging more often than those without children. Those from households with 
children tended to report not feeling a sense of belonging due to being new in the area, while 
those from households without children tended to want to stay to themselves on purpose. 
Finally, Hispanic and Black participants mentioned feeling a lack of belonging more often 
than White respondents. Hispanic respondents who mentioned feeling a lack of belonging 
tended to keep to themselves on purpose, while Black respondents who expressed this 
feeling tended to be new in the area. 


Civic engagement 


Interview participants most often discussed civic engagement in terms of voting—whether 
participants voted and whether they felt voting was effective or worthwhile. 


Voting behavior. For some participants, the system could be changed—or at least one’s 
voice could be counted—through voting. About a third of the participants explicitly said they 
were regular voters. One in ten participants said that they do not vote, and a few mentioned 
that they cannot vote due to citizenship status. 


The most commonly cited motivation for voting was the feeling that it was important to do 
so. One participant stated, “I think it’s important that we have a say-so in what goes on in our 
communities. I feel like I’m a big voter, so I feel like if you don’t vote, then you really don’t have 
a reason to complain about a lot of things.” (4399) Another noted, “So if you want something 
to change, if you don’t go and vote it’s not going to change.” (7481) 


For those who explicitly said that they did not vote, a few made that choice because they did 
not want to engage in politics: “I don’t vote really. I try to stay out of it, I just stay out of it. If 1 
have to, if someone says, well you need to vote, and if it’s going to affect my freedom or 
something like that, I'll do what I have to do, whatever. But as far as me wanting to, or having 
an involvement in it, no, no, I really wish not to. Because I just don’t take sides.” (6535) Others 
did not vote because they tended not to like any of the candidates. A few participants were 
against voting for religious reasons. 


Participants from households without children reported voting more often than those with 
children. In addition, participants who did not have a bachelor’s degree reported that they 
did not vote more often than those with bachelor’s degrees or higher. 


Attitudes about voting effectiveness. Several participants, including both voters and 
nonvoters, had doubts about whether their vote counted or was effective. These participants 
tended to feel like their vote did not make a difference in the government’s decisions, either 
because of distrust in the voting process or a lack of responsiveness from the government 
vis-a-vis the electorate. For instance, one participant, who reported voting occasionally, 
stated: 


I don’t know. I know people say that if you don't vote, you shouldn't have a say- 
so, but I feel like no matter how you vote, it’s going to go any way the government 
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wants [it] to go, you know, how they want it to go anyway. And that probably 
might not be the right frame of mind of thinking, but I do think that sometimes. 
I feel like it doesn’t matter how people vote, that they’re still going to do what 
they want to do anyway. (7503) 


A few participants believed that their vote and political participation counted more at the 
local level than at the federal level. For example, when asked if she felt her vote counted, one 
participant stated, “Locally, yes. ... Nationally, no. ... In a presidential election, not really.” 
(7471) Another participant who did not believe that voting “goes anywhere” mentioned that 
she was trying to get more involved in local politics: “I’ve been trying to start doing that lately, 
because I definitely ... it’s important to do, especially local[ly]. You have all the say. If you don’t 
like something that’s going on in your city, then you have to do something about it.” (4639) 


Some participants who expressed doubts about the effectiveness of voting also stated that 
voting was important to them, suggesting that they still hoped voting would make a 
difference despite their doubts. As one participant said, “/Voting is] very important to me. It’s 
one of those things where you hope that it still makes a difference, but you don’t think it does ... 
But I did vote, and I voted in the last four elections, and I will keep voting. Because I feel like if I 
don't vote, I feel like I haven’t voiced my opinion.” (5103) 


Non-Hispanic participants tended to express doubts about the effectiveness of voting more 
often than Hispanics. Similarly, White participants believed that their vote did not count 
more often than participants of other racial groups. In addition, participants with some 
college, as well as those with bachelor’s degrees, tended to believe that their vote did not 
count more often than did those with high school or graduate degrees. 


3.2.2 Education 


Because the NHES focuses on education, interviewers asked participants for their opinions 
about a variety of education topics. 


Importance of education 


Almost all participants believed that education was important, regardless of whether they 
had school-age children. For some participants, it was difficult to explain why education was 
important; they simply stated that it was. For other participants, the reasons why education 
was important generally fell into two areas. 


Increases employment options. First, some participants thought of education asa 
necessary prerequisite to better paying jobs. This reason was often mentioned by 
participants with lower household income or without a high school diploma, with many 
stating that they did not want their children or grandchildren to struggle with limited 
job options or low hourly wages. An example of such perceptions was, “Well, honey, the way 
the world is, you’ve got to have education, and be smart, to get anywhere. These five-and-dime 
jobs are nothing. If you [luck out and] get a job ... you've got to know how to count and all that 
... 50, you've got to have education to get where you're going.” (7597) 
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Makes you a better person. Second, some participants thought of education as a crucial 
component to being a well-rounded person or a contributing and active member of society. 
These participants believed that education was a pathway to understanding the world and 
how it operates. One participant stated, “I believe that in the long run it’s got to be more where 
the student, male or female, feels comfortable. Gets the education they need to get, the 
experiences. I’m a very big advocate in being grown up, do something with your life, and become 
a productive member of society. That’s very, very important to me.” (5449) 


Almost all participants who had children of any age believed that supporting education was 
a sign of good parenting. According to participants, parents who do not help their children 
stay in school or excel in it are actively reducing their children’s future opportunities and 
intellectual growth. As one participant noted, “It’s like you can’t expect a teacher to really have 
too much interest in your child ifyou have none. You can’t expect the school system to have any 
interest in your child if you have none. I feel like parents should take more interest in their 
children besides just dressing them up.” (5265) 


Quality of education 


Participants had mixed opinions about the quality of education in general. Some believed 
that schools had improved since they were younger; others believed the quality had 
declined. Likewise, some participants felt positively about the current education system, 
while others believed it did not adequately prepare youth. 


Of those who had negative opinions, participants often noted that schools are focusing too 
heavily on math and science to the detriment of arts and civics or skills-based classes, such 
as woodworking or home economics. As one participant noted, “Kids are not taught how to 
live. There are classes where we learn how to do taxes. There’s home ec and stuff. How do people 
grow up and not know how to cook?” (5707) Additionally, some noted that the education 
system is flawed because it focuses too much on standardized testing. One participant said, 
“All the tests that that they do, standardized testing ... And all the curriculum is based around 
what’s going to make you prevail on those tests.” (4289) 


Participants also had mixed opinions about the quality of local schools and the education 
their own children were receiving. Some were pleased with the quality. As one participant 
put it, “His teachers, they[‘re] great. But my younger son, his teachers are more hands-on. He’s 
in a smaller classroom, so I’m happy with his teacher.” (7057) Many of these participants 
praised the fact that their children were exposed to a diverse learning environment. For 
example, one participant stated, “We like the public school system because I want them to get 
the diversity and the inclusion of public school.” (7303) Participants who had a bachelor’s 
degree or more as their highest level of education tended to have a more positive opinion of 
their children’s public schools or the schools in their general area. 


Others found the quality of local schools to be lacking. One participant explained: “Especially 
with all the [overcrowding] of the classroom, which is, to me, the biggest problem. How are you 
going to have 40 kids in a classroom? Teaching class, how can you expect to teach?” (7715) 
Many participants, both with and without school-age children, believed that there was 
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unequal financial support across public schools. According to these participants, schools 
with fewer resources offered fewer and less rigorous classes and extracurricular activities. 
One participant observed how school offerings had declined in the recent past: “The quality 
of the education, they don’t have the courses, for example, foreign language, they don’t have 
that. Extracurriculars, they don’t have [it] or they have to pay for it.” (7765) One participant 
noted the unequal division of funding for schools in their area: “Even in [urban city], you can 
tell the nicer schools that are funded by taxes, and they’re really great schools, and the 
[crummy] school|[s], some of the schools that I went to coming from a single income with two 
kids [...] why can’t they improve something that’s been there since, I don’t know, the freaking 
‘70s?” (4639) 


The Department of Education 


About one-fifth of the participants offered their opinions about the Department of Education. 
Of those, the majority were negative. These opinions were often entangled with participants’ 
broader feelings about the education system or the federal government. White participants 
reported having negative views of the Department of Education more often than participants 
from other racial/ethnic groups. Final NHES nonrespondents also reported negative views 
of the Department of Education much more often than late respondents. 


Participants cited a variety of reasons for negative views of the Department that mirrored 
many of their overall concerns about the education system, including insufficient funding for 
schools and lack of school safety. Others expressed concerns around inequities in the school 
system and ineffectual bureaucracy. For instance, one participant believed the Department 
had a role in the low salaries of teachers, saying, “And you're telling me you’re paying 
[correction officers] 90 plus a year and you're paying a teacher who’s educating our students 
and children less than what, less than 30 a year, 40 a year? Get out of here, you guys are out of 
your minds. So I just, I’m all about education. Because I think they got it twisted.” (6535) 


A few participants did have positive views of the Department of Education. One participant 
gave the Department of Education credit for recent improvements in the education system, 
noting: 


‘I do think [the Department of Education has] worked to improve the education 
system. And I’m seeing even down to the elementary and middle school, high 
school, [and] the younger kids’ levels... that they’re finally changing things back 
around. And it’s not as made up. Like history class now is not what I learned in 
history. It’s not the same history. ... I think that they’re starting to slowly turn 
that back around, which is awesome, because by the time I have kids, Iwant them 
to be learning the right things.” (4209) 


Another participant stated that the Department of Education was committed to its mission 
and was not used for political purposes: “] trust them. ... Not going to use the job [at the 
Department of Education] to run for president. [I] feel the people are more committed to the 
goal of education.” (4129) All of the participants who had positive views of the Department 
of Education were from households with children. 
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Finally, of those who addressed the topic, a small number did not have an opinion about the 
Department of Education or said that it was not relevant to them. For example, when asked 
if the Department of Education affected her life in any way, one participant responded, “No. 
Not really, because like, [I] don't have any more kids in school so it doesn't bring anything to 
my brain. Maybe when [my great-granddaughter] starts going to school, then.” (6331) In 
addition, a handful of participants noted that they were not familiar with the Department's 
function or purpose. Hispanic participants did not report an opinion of the Department of 
Education more often than White or Black participants, although when they did share their 
thoughts, they were typically negative. 


3.2.3 Privacy 


Concerns about privacy and the safeguarding of their data may deter people from 
participating in surveys (Singer and Presser 2008; Robertson et al. 2018). In this study, 
almost all participants discussed privacy at some point during the interview, although how 
they defined privacy and their level of concern about privacy varied. Participants generally 
defined privacy as protecting personal information or maintaining distance or boundaries 
between themselves and others. Many participants conceptualized privacy in both ways. 


Protecting personal information. Almost ll _participantsdefined privacy 
as protecting personal information and took some measures to safeguard their personal 
data, whether virtual or hard copy. These measures included not giving out their Social 
Security number or full birthdate, changing their password frequently, connecting to the 
Internet with a virtual private network (VPN), and using an identity theft protection 
service. For example, one participant said that he does not “put private information such as 
address or Social Security number on [social media]. The most I would put would be my e-mail 
and maybe my phone. But besides that, I wouldn't put anything there.” (4655) Another said she 
protects the following items: “name, the social, the date of birth, where I work, how much I 
earn.” (5305) 


Maintaining distance or boundaries. Many of the participants also talked about privacy in 
terms of maintaining distance from others or distrusting other people or organizations, 
including the government. Some participants talked about this in terms of wanting to fly 
under the radar. One described himself as a “quiet guy” who does not want people changing 
the way he lives his life or keeps his home (5247). Others discussed it in terms of wanting to 
keep to themselves. When asked how well she knew her neighbors, a participant replied: 
“Let’s put it this way: I say hi to them, we’re friendly and all that, we do everything we can for 
everybody, but everybody minds their own business. That’s the best way to do it. I don’t call 
them neighbors. I don’t go to their houses, you [know] what I mean. To me, it’s an invasion of 
privacy.” (7597) 


Most participants who talked about privacy in this way mentioned knowing people, 
including themselves, who had had negative experiences that made them distrusting or 
suspicious of others. Some discussed being victims of scam attempts. Others noted negative 
encounters with institutions like the police or the local elections board. One participant 
shared her experience: 
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I’ve been a victim of identity theft. I don’t know how my information got taken, 
but I went to jail because somebody stole my identity. So, I’m not very willing to 
[share my information]. This was when I was [younger]. I went to jail. [The 
police] said I had a whole bunch of [parking] tickets, and I never even had a car. 
(7057) 


Several other participants pointed out that it was an aberration for them to even do an 
interview for this study because they frequently do not let people into their homes or answer 
questions about themselves. As one described, “Imagine yourselves not here and just me 
sitting here. That would be my ideal.” (6535) Likewise, one participant commented at the 
completion of the interview that she only participated because the team looked very kind 
and resembled her granddaughters (7597). 


Privacy by participant characteristics. Participants who ultimately responded to the 
NHES appeared to be less concerned than final nonrespondents to the NHES about privacy. 
Participants from households with and without children did not substantially differ in their 
discussions around privacy, except that participants from households with children often 
talked about how they were even more protective of their children’s information. When 
looking at the NHES paper screener, one participant noted, “J would be fine with my child’s 
age and their gender, but I wouldn't want their birthdate or full name—or even their initials. I 
feel like you can't really be too safe.” (4277) 


There were several differences by race and ethnicity in the types of privacy concerns that 
participants mentioned. For example, Hispanic and Black participants talked more 
often about privacy as not giving out one’s Social Security number and not using credit cards, 
whereas White participants talked more often about privacy on social media. Black 
participants tended to be more skeptical in general, including of online banking. A few 
Hispanic participants who were recent immigrants reported being afraid of scams or identity 
theft that could have consequences for their legal status. One participant described these 
fears: “Sometimes those gentlemen [scammers] come here to tell us, the way sometimes they 
talk to us, they say that I must give them the Social Security number, and I told them that no 
one was going to give [it to] them. Those are the fears that we sometimes feel.” (4187) 


Participants with less education tended to report being a very private person more often that 
those with more education. For participants who did not have a bachelor’s degree, privacy 
generally was related to keeping their distance from their neighbors rather than to 
government or political issues. In particular, they voiced more concerns about issues with 
protecting their identity than did those with a bachelor’s degree or more. 


Degree of concern about privacy 


About half of the participants explicitly stated that they were not concerned about privacy, 
either because they felt only people with something to hide needed to be concerned or 
because they assumed, given advances in technology, there was no privacy. Only a few 
participants said that their reason for not being concerned about privacy was that they 
believed their data are secure and their connections to or relationships with others are 
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generally safe and trustworthy. One participant said that she was “pretty much an open book” 
and “didn’t mind sharing.” (6587) Another said that she trusted both the government and 
others, and “if they want the information, I can give [it].” (6377) 


However, most people who said they were not concerned about privacy believed their 
information was already out there so being concerned was pointless. They believed that 
their data were freely available to other people and institutions, including the government, 
making statements such as, “I don’t think [information] can be safeguarded. Look at all the 
hacking that’s going on. For every safeguard that somebody puts up there, there’s someone else 
looking for a way to get through that.” (7355) Another who said she was not concerned about 
privacy shared that “I think Big Brother is watching. Every second, every day. They can listen 
to our phone. Does that make it wrong? No, I think for the safety and security of our population, 
they have to, to some degree, pay attention.” (7015) 


Some participants stated thatthey had moderate concerns about privacy. This group 
worried about their information falling into the wrong hands or that their information was 
not well protected. This group tended to talk more about targeted marketing and digital 
traces than those who were not concerned about privacy. One participant described it this 
Way: 


I feel like any information could be just so dangerous. Right now, I might not feel 
like it’s dangerous, but then in 5 years down the road, something will come up 
and “Oh I should have been concerned about that.” Anytime I use a rewards 
[card] at a store, they know what I am buying. All of that information, where is 
it going? It’s not just to give me good coupons for the next week; it’s going 
somewhere. | try not to think about that stuff. (5103) 


Another shared that he was authorized to help local municipalities with data breaches and 
ransomware attacks (5203) He believed that “they don’t protect [anything], at least on the 
local and state level.” 


A few participants were extremely concerned about privacy and took more extreme steps to 
protect it, including not using social media or cell phones, not using banks or credit cards, 
and burning their mail (see section 4.2.1 for more details). One participant, who burned his 
mail and declined to be recorded during the interview, said that the government has access 
to everyone’s information and can listen to and transcribe our conversations through 
Amazon’s Alexa service (7219). 


There were few subgroup differences in terms of the issue of degree of privacy concerns. 
White participants tended to believe that government could access all their information 
already more often than participants of other races and ethnicities. Among those with at least 
some degree of concern about privacy, Black and Hispanic participants tended to express 
more concern as compared to Whites. There were no patterns across subgroups in terms of 
extreme privacy concerns. 
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3.2.4 Government 


Given that NHES is a federal government survey, individuals’ attitudes toward the 
government may play a role in their decision about whether to respond to the survey. 
Interviewers asked participants about several government-related topics, such as their 
general attitudes toward the federal government and what role, if any, they felt the 
government should have in conducting data collections. 


Patriotism 


A few participants mentioned that they considered themselves to be patriotic, fortunate to 
live in the United States, or felt a duty to support their country. Despite these feelings, these 
individuals raised significant concerns about the country. For instance, one participant was 
concerned that political leaders have their self-interest in mind rather than the country’s 
stated ideals, saying that, “I served this country. I love this country. I’m very patriotic. I love 
what this country says it stands for, it’s just most days I don't think the people who are in charge 
really stand for those things.” (4289) Another participant believed that despite doing the best 
they can, government leaders in the U.S. are not doing a good job: “Government, on the other 
hand, they do what they can. I personally don’t think they do a very good job, but they do the 
best, I guess, they can. But hey. I’m in America, one of the best countries in the world. I feel very 
blessed, and I could be somewhere worse than here.” (6221) These negative feelings were only 
mentioned by NHES final nonrespondents; otherwise, views on patriotism did not tend to 
vary by participant characteristics. 


General attitudes toward the federal government 


About three-fifths of the participants explicitly shared their thoughts on the federal 
government. Few had positive views. Those who did expressed support for the 
administration at the time of the interviews in 2019. 


Some participants had neutral or mixed views of the government. Those who had neutral 
views tended to fall into two groups: people who described themselves as anti-political and 
those who did not have issues with government as long as their personal life was going well. 
For example, a participant who expressed neutral views stated that she did not keep up with 
the government enough to offer an opinion: “I don’t keep up with politics like that, so I really 
don't have a voice in it to say, really, if it’s good.” (7715) Other participants did not have 
concerns about the federal government as long as they were personally doing well. For 
instance, one participant stated, “To be honest, I don’t even care about the government. They 
do what they're going to do. Even though we try to stop, they’re still like, they’re going to do 
what they’re going to do. ... So I really don’t care about the government, as long as I’m living 
my life, living happy, trying to do everything good the right way, me—I don’t care.” (6081) 


Those with mixed views of the federal government tended to have negative views of it but 
with some sort of caveat that government has a purpose and to some extent works well. For 
instance, one participant stated that, “ believe the government serves a purpose, and we need 
a government. I absolutely believe that. Do I believe that every individual and the head of our 
government are competent? Or productive? No.” (7015) 
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Participants with negative views of the federal government voiced a variety of concerns. For 
example, some believed that the government was driven by greed rather than by public 
interest and that some government agencies were inefficient—although some of these 
participants acknowledged that some workers and federal agencies were likely trying to do 
their best. Others did not feel represented by the government, stating that it was not 
responsive to the people and was not representative of the diverse U.S. electorate. One 
participant noted, “/The government] is not really representative of who people are out in the 
real world, and [the] people in charge tend to not fight for that.” (5611) Finally, some 
participants felt the federal government interfered too much in their lives, either through 
collecting too much information about individuals or interfering in state and local 
government. As one participant shared, “Government shouldn't be there to tell you how to live 
your life, right? ... So, as far as from a federal level, I believe ina smaller government that’s less 
intrusive into your life, right? The local level plays a much different role.” (7329) This 
perspective became especially clear when discussing what role, if any, the government 
should have in data collection. 


Government access to and collection of information 


Just over a third of the participants spoke explicitly about government access to and 
collection of information about individuals. Some of these participants reported being 
comfortable with the federal government collecting information through surveys. Some 
thought that they were helping the government by sharing their opinions, with one stating, 
‘I don’t think [participating in government surveys] is a bad thing, because how would they 
know what to perform if they don’t know what people want them to do?” (6587) Other 
participants had positive attitudes about the government collecting data even though they 
did not understand the purpose of it: “J can’t remember exactly what they use it for, but I know 
it’s important and it’s a good thing to have. It’s not a bad thing. I don’t see anything bad about 
it. It wasn’t no questions on there that I thought was vulgar or out of the way or nothing like 
that.” (5703) Yet other individuals felt that government surveys were fine as long as 
individuals’ privacy was maintained through the process. For example, one participant 
thought that the federal government needed to collect some information about the 
population but should not pry into individuals’ personal lives: 


I think [the government] need[s] to know what people are doing in terms of work 
habits. That way they can guide people towards careers and habits that benefit 
the country as a whole. ... But prying too much into personal lives, like I don’t 
know, tracking telephone calls or tracking websites and things like that ... Idon’t 
think it’s necessary for them to control the internet or collect data based on that. 
(5479) 


A larger number of participants expressed concerns or had more negative views on the topic. 
The main themes in these views are summarized in the rest of this section. 


Not responsive enough. Some participants felt that the government did not take enough 
action based on the results of surveys. An example of this sentiment was, “[Government data 
collection] is a good and bad thing. Well, I mean, they trying to perceive what’s going on, that’s 
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the good thing about it. But once they do see what’s going on, I don't feel like ... [there’s] too 
much of an effect on the community.” (7057) Another participant shared, “I think it’s important 
if [the government] actually paid attention to [surveys]. Because it’s one thing to go out and 
collect the data, it’s another to actually process it and pay attention to what the data is telling 
you and making changes accordingly. That’s the part that [the government] should invest [in], 
in my opinion.” (4209) 


Government already knows. Several of the participants who had negative views of 
government data collection believed that the government already has individuals’ 
information. As a result, they did not understand why government surveys were necessary. 
These participants tended to believe that their information was “already out there” in some 
way and that the government agencies shared this information among themselves. 


Some participants believed that the government used Social Security numbers to collect 
information on different aspects of individuals’ lives. When asked how she felt about the 
government collecting information, one participant noted, “Even if they don’t ask, they know 
it already. With your Social Security, you know they got you, so ... | know before they ask, they 
know it. I don’t know why they keep asking, they know it. You know that. They control us.” 
(4271) Another participant expressed similar views, but in relation to her children. The 
participant stated, “They have [my information], I’m sure. I’m sure. But also, I don’t know 
because my children are not technically in school, they’re not enrolled, their names aren’t 
already out there. But they’re living beings, and they have a Social Security number, so I’m sure 
that their information is out there. I’m sure that they have it.” (4277) 


Other participants believed that personal information collected by one agency on a tax form 
or school registry (such as children’s names and ages) also would be available to other 
agencies in the government. As one participant who was reluctant to give out his children’s 
personal information on the NHES screener noted, “I mean if it’s the government you know 
they'd see on my tax returns or something. ... I kind of think a lot of that information would be 
known. ... I think the government, you know, could obtain it anyway legally. Get it off my tax 
returns or there’s probably, you're probably registered in some national school registry. I can’t 
believe that they need to know my kids’ names.” (7055) In addition, when discussing whether 
she felt comfortable sharing personal information on her children in the NHES, one 
participant said, “I would think that they would have their information already in the school 
system. But then you’re looking for the month that they was born, the year and date. This thing 
personal. ... They should know this already, right? The school system?” (5409) 


A few participants believed that the government had access to their information through 
more extreme means, such as online tracking and surveillance. One participant believed that 
the government knows every move you make and that no information is truly private (7063), 
while another believed that the government had her phone tapped (5265). Another shared: 


I just feel like all our information is everywhere and everyone’s watching you and 
listening to you and you're being tracked at every turn. Every purchase you 
make, every phone call you make. Every, I don’t know, every time you turn on 
your Google Maps. I feel like any government entity that’s tracking me or spying 
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on me is probably, at this point, is probably not good. Not a good thing and what 
are they going to use my information for in all this? ... Before it mattered what 
branch or what form and all of this, but now I hear the government and it’s like 
this one big nasty umbrella of government stuff. (5103) 


Concerns about data use. Most of the participants who expressed concerns about the 
government’s collection of information through surveys were skeptical of how the 
government would use their information. For instance, one participant noted that she was 
not inclined to complete government surveys because she was “not sure what [they] will do 
with this information.” (4365) Further, a couple of participants cited cases of hacking against 
the government as examples of the U.S. government’s vulnerabilities, stating, “Hackers can 
get anything they want. They hack all the way to Washington, D.C.” (5515) Another participant 
did not trust the federal government to provide accurate results or interpretations of survey 
data (7355). Additional concerns included that the government may be selling personal 
information to corporations and using surveys to target people for immigration raids and 
deportations. For instance, one participant thought that the government sold people’s 
personal information, just like corporations did (7063). Further, a few participants, both 
immigrants and nonimmigrants, expressed fears that sharing information with the federal 
government through surveys or the Census may lead to deportation orders. One participant 
said, “It depends on the information that they’re collecting and how they’re using that 
information. [If] they’re going to be using it to pass tougher immigration laws or something like 
that, I would be a little weary of answering questions like that.” (5103) 


Concerns about legitimacy. Finally, a few participants worried that surveys described as 
being from the government were not actually from the government and might instead be 
scams to obtain their private information. For instance, one participant who thought it was 
fine for the government to collect information through surveys stated that she would check 
online or call her friends to make sure that government surveys were legitimate before filling 
them out (6377). Another participant noted that people might be wary of surveys because 
police have been known to use similar scams in certain neighborhoods to arrest people. 


A lot of people don’t take the time [to respond to surveys]. ... I! mean you got cops 
that are using scams like this to reel in people... that haven't paid their taxes or 
that are outstanding so they can get arrested. People just don’t fill stuff out. 
They'll be like, “Hell No!” Government, nope, government pfft. It all goes in the 
trash. I’ve seen it. (6535) 


Another participant noted that having a person come to their door, show a badge, and explain 
the purpose of a government survey would make them less skeptical of filling it out (6587). 


Government access to and collection of information by participant characteristics. 
Late respondents tended to have positive or mixed attitudes toward the government 
collecting information. Final nonrespondents had a wider array of views on government data 
collection that included negative views. However, a few final nonrespondents stated that 
they were fine with government-sponsored surveys as long as they were actually from the 
government. 
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Participants from households with children reported negative views of government access 
to and collection of information, as well as concerns about sharing information with the 
government, more often than participants from households without children. Several 
participants from households with children were concerned with sharing their children’s 
personal information ona survey or believed that it was already available to the government. 


Compared to non-Hispanics, Hispanic participants reported more often being afraid that 
information gathered through government surveys would be used to target them for 
deportation orders or other immigration-related issues. White participants reported having 
negative views of government collection of information more often than other groups. They 
also thought that their personal information was already accessible to the government more 
often than other groups. 


Participants with some college or less, particularly those with high school degrees, reported 
believing that the government already had their information more often than those with a 
bachelor’s degree or above. In addition, participants with some college or less believed that 
the government was actively surveilling them more often than those with bachelor’s degrees 
and above. 


The United States Census 


Because the U.S. Census Bureau is the data collector for the NHES, the Decennial Census was 
discussed in several of the interviews, with over three-fifths of the participants sharing their 
views. Participants’ views of the Census may be important given current use of the Census 
Bureau for NHES data collection and the Census branding that is used in communications for 
the survey. 


Participants mentioned a variety of opinions on the purpose of the Census, such as producing 
a count of the U.S. population, determining allocations for government services, and 
determining political representation for districts. When mentioned by the interviewer, 
several participants were surprised at the fact that the Census was conducted every 10 years. 
Irrespective of the schedule, a few participants mentioned that receiving NHES materials 
from the Census Bureau in a non-Census year was confusing to them (see section 5.2 for 
more information), with one participant noting that she discarded the NHES materials 
because she was skeptical of receiving materials marked as from the U.S. Census in 2019, a 
non-Census year (4025). 


Participants expressed a variety of attitudes toward the Census. Although several of the 
participants felt that the Census was important and some stated that they intended to 
complete it, some voiced concerns about the proposed citizenship question and doubts about 
the Census’s accuracy or lack of benefit to them personally. For example, one participant 
stated, “Really, I don’t see a benefit. You answer things you don’t want to. Some people are 
honest, others are not and so ... | don’t know what their real purpose may be. But, for me, I don’t 
get anything from it.” (4711) Another participant echoed this view by saying, “There’s no 
direct benefit to it. There’s no public benefit that I tend to see with the Census. And I think that 
a lot of the time, [the] numbers are so skewed that [the] statistics are kind of useless.” (5203) 
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Participants’ attitudes toward the Census tended to vary according to participant 
characteristics. Late respondents to the NHES expressed much more enthusiasm about the 
Census than final nonrespondents. When discussing the Census, Hispanic participants and 
those from Spanish-speaking households mentioned being afraid of consequences to their 
immigration status more often than non-Hispanics. Despite this, both groups reported 
intending to complete the Census more often than non-Hispanic participants or English- 
speaking households. In addition, participants with some college or less thought that the 
Census did not have a benefit for them more often than those with college degrees or higher. 


3.3. Address Observation Findings 


The final section of this chapter presents address observation findings that shed light on the 
characteristics of the sampled addresses and the individuals residing in them. In addition to 
being limited to observed addresses that were final nonrespondents to NHES:2019,° they 
also exclude a small number of addresses that observers determined to be nonresidential. 
Addresses with UAA outcomes also were excluded from these analyses; the characteristics 
of those addresses are the focus of section 7.2 in chapter 7. 


We also repeated several of the observation analyses with the sample restricted to those 
addresses that completed an interview. Because the results were relatively similar to those 
for the larger group of observed addresses, they are not discussed here (but see tables A.3.4 
and A.3.5 in appendix A). 


3.3.1 Structure Type 


As shown in table 3.2, 50 percent of the addresses were single-unit structures, such as single- 
family homes. Thirteen percent were part of attached structures (either duplexes, 
townhouses, or rowhouses). The remaining 37 percent were apartments. Most of these 
addresses were in low-rise apartment buildings (with three or fewer floors), but some were 
in mid-rise or high-rise apartment buildings (with 4 or more floors). In a small number of 
cases, observers were unable to determine the structure type (for example, because the 
building was under construction). 


9 See appendix C for the results of a sensitivity analysis that drove the decision to exclude late respondents (that is, addresses that 
responded to the screener after the fourth screener package was sent) from these analyses. 
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Table 3.2. Number and percentage distribution of structure type observation for 
observed, residential, nonrespondent addresses: 2019 


Number of observed, Percentage of observed, 

residential, nonrespondent residential, nonrespondent 

Structure type addresses addresses 
Total 530 100.0 
Single-unit 260 49.7 
Duplex 30 6.0 
Townhouse or rowhouse 40 6.8 
Low-rise apartment! 130 24.9 
Mid-rise apartment! 40 7.2 
High-rise apartment! 30 5.1 


Could not determine $ £ 


+Reporting standards not met. There are too few cases for a reliable estimate. 

1Low-rise apartment buildings are those with 1 to 3 floors. Mid-rise apartment buildings are those with 4 to 6 floors. High-rise apartment buildings are 
those with 7 or more floors. 

NOTE: Structure type observations were not collected for addresses that could not be observed (e.g., cannot locate the address). Addresses that ended up 
responding to NHES:2019 and addresses that had at least one undeliverable as addressed (UAA) NHES:2019 mailing were excluded from this analysis. 
Addresses that were observed to be nonresidential were excluded from this analysis. Sample sizes are rounded to the nearest 10. Percentages are rounded 
to one decimal place but have not been changed to reflect sample size rounding. Details may not sum to totals because of rounding. 


SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Surveys Program (NHES), 2019. 


3.3.2 Household Member Characteristics and Attitudes 


Observers also collected a variety of observations to provide information about the 
characteristics, attitudes, and interests of the household members. This information was 
only collected for addresses that observers determined to be currently occupied. As shown 
in table 3.3, the observed prevalence of these characteristics and attitudes varied greatly. 


Table 3.3. Percentage of observed, occupied, residential, nonrespondent 
addresses with observed household attributes: 2019 


Observed household member characteristics, Percentage of observed, occupied, residential, 
attributes, and interests nonrespondent addresses! 


Household member characteristics 


Presence of children? 21.4 
Speaking a language other than English? 5.5 
Household member attributes and interests 

Privacy or security concerns* 31.2 
Outdoor living 24.6 
Patriotism® 7.7 
Welcoming decor’ 6.6 
Internet or television connectivity® 6.8 
Community involvement? 3.6 
Importance of religion?® 1.8 
Pride in education!! 1.4 
Other outdoor decor” 24.6 


1Addresses that ended up responding to NHES:2019 and addresses that had at least one undeliverable as addressed (UAA) NHES:2019 mailing were 
excluded from this analysis. 

2Addresses where children are thought to be present are those where observers identified indicators that suggest the children live in the sampled unit. 
Examples include toys, bikes, car seats, strollers, outdoor swings/play sets, and child finder stickers for firefighters. 
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3Addresses that speak a language other than English are those where observers identified indicators that people in the sampled unit may speak a language 
other than English. Examples include flags for non-English-speaking countries, in-home business signs, yard/window signs, or bumper stickers written in 
another language at the sampled unit; or a large number of flags, signs, or bumper stickers in (or including) another language in the surrounding 
neighborhood (even if indicators are not observed at the sampled unit). 
4Addresses with privacy or security concerns are those where observers identified indicators that privacy or security is important to those living in the 
sampled unit. Examples include surveillance cameras, driveway gates, security company signs or stickers, and "No trespassing" signs. For multi-unit 
buildings, indicators that focused on the larger building in which the unit was located, such as entry buzzers or fences, were not included because they 
could be not directly associated with the sampled unit. 
5Addresses that value outdoor living are those where observers identified indicators that people in the sampled unit spend time outdoors. Examples 
include patio furniture, porch swings/benches, swing sets, sporting goods, and grills. This variable was added during data processing based on patterns 
observed in write-in responses. 
6Addresses that value patriotism are those where observers identified indicators of American national or state-specific patriotism. They also include those 
with indicators of current or past involvement with the U.S military or pride in the U.S. military. 
d 
c 


7Addresses with welcoming decor are those where observers noted the presence of an object that welcomes visitors to the sampled unit, such as a 
welcome mat or welcome sign that explicitly says “welcome.” This variable was added during data processing based on patterns observed in write-in 
responses. 
8Addresses with Internet or television connectivity are those where observers noted the presence of a satellite, cable, DirectTV dish or other electronic 
equipment, or a sign that indicates television or internet connectivity at the sampled unit. This variable was added during data processing based on 
patterns observed in write-in responses. 
°Addresses with community involvement are those where observers identified indicators that people in the sampled unit are involved in the community. 
Examples include political candidate signs or indicators of being involved with charities, kids’ sports/clubs, or neighborhood associations. 

10Addresses that value religion are those where observers identified indicators that suggest that religion is important to the people in the sampled unit. 
Examples include “Bless this House” signs, religious figures or statues, or Mezuzahs. This variable was added during data processing based on patterns 
observed in write-in responses. 
11Addresses with pride in education are those where observers identified indicators that education is important to the sampled unit. Examples include 
school, college and/or university flags or stickers; honor roll bumper stickers; or other indicators of pride in a child’s school. 

12Addresses with other outdoor decor are those where observers noted the presence of items used to decorate the exterior of the sampled unit. Examples 
include gnomes, flamingos, yard art, decorative flags, wreaths, Easter decorations, water features, or windchimes. This attribute focuses only on 
decorative items and does not include items that represent another household attribute shown in the table (e.g., American flags were coded under 
patriotism only). This variable was added during data processing based on patterns observed in write-in responses. 

NOTE: Percentages represent the percentage of addresses for which the attribute was observed. Household attributes were not collected for addresses 
that could not be observed (e.g., cannot locate the address) or for which only a partial address observation could be completed (e.g., can observe the multi- 
unit building but cannot gain entry to observe the sampled unit). Household attributes also were not collected for addresses that were observed to be 
nonresidential or temporarily or permanently vacant. The rounded eligible sample size for observed addresses is 440. Details may not sum to totals 
because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Surveys Program (NHES), 2019. 


Household member characteristics. Observers concluded that children were living at 21 
percent of the addresses. In a handful of addresses, observers felt there was evidence that 
children did not live in the address. However, for most addresses, there was not enough 
evidence that was visible from the exterior of the unit to definitively say that children did (or 
did not) live there. Evidence of speaking another language—either by residents at that 
address or in the surrounding area—was noted for 5 percent of addresses. Most of the time, 
the observed language was Spanish. 


Household member attitudes and interests. The most commonly observed attitudes and 
interests were privacy or security concerns (31 percent) and an interest in outdoor living 
(25 percent). Most other specific attitudes and interests were much less commonly observed. 
Only 7 to 8 percent of addresses had evidence of patriotism, welcoming decor, or internet or 
television connectivity. And only 4 percent or less had evidence of community involvement, 
the importance of religion, or pride in education. However, observers did note the presence 
of other outdoor decor for 25 percent of the addresses; this includes any decorative items 
that were not also coded as indicators of other household attitudes or interests. The 
relatively higher prevalence of these items suggests that some households do place 
decorative items outside of their homes—but the exact items they choose to display may not 
shed light on their interests and values. 


Observers also provided notes about how they decided whether there was evidence of these 
characteristics, attitudes, and interests at each address. Exhibit 3.1 summarizes the types of 
evidence reported by observers. 
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Exhibit 3.1. Types of evidence reported by observers, by observed household 
member characteristics, attitudes, and interests 


Characteristics, attitudes, 
and interests 


Presence of children 


Types of evidence reported by observers 


Children’s items outside the unit (e.g., toys, child-sized shoes, swing sets) or 
in the window (e.g., child’s art taped to the window); child’s voice coming 
from the unit; visual confirmation of child (e.g., playing in the yard) 


Speaking a language other 
than English 


Privacy or security concerns 


Another language coming from the unit (e.g., voices or music); another 
language spoken by several people in the neighborhood; written indictors 
of another language (e.g., street signs, business signs, posted notices) 


Yard sign or window sticker for security companies; sign indicating desire 
for privacy (e.g., “no soliciting,” “private property”); gate or fence; security 


camera 

Outdoor living Outdoor furniture; grill or other outdoor cooking-related items; sporting 
equipment 

Patriotism American flag or state flag; item with image of American flag (e.g., license 


plate or bumper sticker); item suggesting pride in U.S. military (e.g., sticker 
for the U.S Marines); other symbolic items (e.g., bald eagle figurine) 


Welcoming decor 


Door mat or sign/banner on the door with statements of welcome (e.g., 
“Welcome,” “Welcome to chaos,” or “Welcome to our home”) 


Internet or television 
connectivity 


Satellite or cable dish or antenna; other electronic equipment (e.g., an 
internet booster); sign for an internet or television company 


Community involvement 


Importance of religion 


Sign supporting local- or state-level political candidates; and sign, sticker, 
or flag supporting community- or health-related causes 


Cross; statue; angel figurine or decal 


Pride in education 


School flag or signs (e.g., “Home of a Knight”); image of school mascot 


Other outdoor decor 


Wreath; potted plants; lighting elements (e.g., lantern, string lights); bird 
feeder or bird bath; water feature (e.g., fountain, man-made pond); 
decorative landscaping items (e.g., wind chimes, garden gnomes, flags); 
holiday decorations 


SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Surveys Program (NHES), 2019. 


A series of subgroup analyses were conducted to explore whether the observed 
characteristics varied for different types of addresses. The results of these analyses are 


discussed in appendix D. 
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Chapter 4. Receiving and Processing Mail 


This chapter describes information collected about how participants receive and process 
mail. By understanding the ways in which participants check, sort, and make decisions about 
opening, keeping, and discarding mail, we can explore factors that may influence 
nonresponse that are not tied to the materials themselves. Identifying the structural and 
logistics challenges participants may encounter around mail delivery and location of their 
mailbox can also shed light on external factors potentially affecting whether households even 
receive the NHES mailings. 


To address these goals, the chapter is divided into two sections. Section 4.1 focuses on 
receiving mail, using data from the observations and interviews to outline where 
participants receive their mail, what kind of items they usually receive, and their attitudes 
and behaviors around checking the mail. Section 4.2 focuses on processing mail, using 
interview data to describe how participants process their mail and the results of an activity 
where participants reviewed an example bundle of mail. Throughout this chapter, we 
conducted interview subgroup analyses using the priority subgroups described in chapter 2, 
with the addition of characteristics that were hypothesized to impact mail-related attitudes 
and behaviors (e.g., number of adults in the household, mail access type, structure type, 
urbanicity). Subgroup findings are reported only for those analyses where notable 
differences arose. 


4.1 Receiving Mail 


This section explores participants’ attitudes, experiences and behaviors related to receiving 
mail. Attitudes about checking and receiving mail may influence individuals’ willingness to 
respond to mail-based survey contacts, such as those used in the NHES. In addition, certain 
experiences and behaviors, such as rarely checking the mail or experiencing mail delivery 
challenges, may influence the ease with which sample members can respond to mail-based 
survey contacts. 


4.1.1 Attitudes About Receiving Mail 


Most participants thought of receiving and checking mail as routine activities not worthy of 
much thought. As one participant summarized, “It’s a part of life. I do it without thinking. It’s 
like eating; when you got to eat, when you got to check the mail, it’s all the same.” (7597) Less 
than 1 in 10 participants expressed frustration with mail, usually in relation to junk mail. 
One participant shared, “I hate mail; it’s, like, the worst. ... Nothing important comes through 
the mail anymore. It’s either e-mailed to me or I am expecting something.” (5103) A few 
participants felt that the U.S. Postal Service would no longer be operating within the next few 
decades. 


4.1.2 Location and Type of Mailbox 


As part of an effort to better understand nonrespondents’ mail checking process, observers 
recorded information about the mail access type of the addresses that were sampled for the 
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observation component of the study.!° As shown in figure 4.1, about 37 percent of observed 
addresses had their mail receptacle attached to their home; 6 percent had a mail slot and 31 
percent had a mailbox (also see table A.4.1 in appendix A). Another 31 percent had a mailbox 
at end of their driveway (20 percent) or across the street or at the end of the road (11 
percent). About 15 percent received mail through a mailbox, slot, or room in their apartment 
building. Additionally, a small number of observed addresses had more than one potential 
way to receive mail (e.g., a mail slot in the door as well as a mailbox attached to the house). 
About 8 percent of addresses had no mailbox or slot in view; in these cases, the observer felt 
he or she had exhausted all potential opportunities to locate the method of receiving mail 
and was not able to identify one. For the remaining 10 percent of addresses, the observer 
was not able to determine the mail access type. This tended to occur at addresses where the 
observer could not gain full access to the property or building (for example, if the observer 
could not gain access to the interior of an apartment building to check for a mail room). 


Figure 4.1. Percentage distribution of mail access type observation for observed 
residential, nonrespondent addresses, by mail access type: 2019 


Percent 
100 
80 
60 
40 
31 
20 = 15 
11 
6 8 10 
0 t 
Mail slot Mailbox Mailbox at the Mailbox across Mailbox, slot, More than one No mailbox or Could not 
attached to endofthe thestreetorat orroomin type of mail — slot in view determine 
the home driveway theendofthe multi-unit access ! 
road building 


Mail access type 


+ Reporting standards not met. There are too few cases for a reliable estimate. 

1This category includes addresses where mail could have been received in more than way (e.g., a house that has both a mail slot in the door and a mailbox at 
the end of the driveway). 

NOTE: Mail access-type observations were not collected for addresses that could not be observed (e.g., cannot locate the address) or were observed to be 
nonresidential or permanently vacant. Addresses that ended up responding to NHES:2019 and addresses that had at least one undeliverable as addressed 
(UAA) NHES:2019 mailing were excluded from this analysis. Details may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Surveys Program (NHES), 2019. 


An address’s structure type appeared to be a driver of its mail access type. Addresses that 
were observed to be apartments were most likely to have mailboxes, slots, or rooms in multi- 
unit buildings (40 percent), while this mail access type was very rare for single-unit or 
attached structures (see table D.3 in appendix D). In contrast, single-unit and attached 


10 As in chapter 3, the observation results presented in this section are limited to final nonrespondent, non-UAA addresses - that is, they 
exclude (1) addresses that ended up responding to NHES:2019 and (2) addresses that had one or more NHES:2019 UAA outcomes. They 
are also limited to fully or partially observed residential addresses that did not appear to be permanently vacant. 
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structures mostly had mail slots or mailboxes attached to the home (47 percent and 54 
percent, respectively) or mailboxes at the end of the driveway, across the street, or at the 
end of the road (45 percent and 34 percent, respectively); only about a quarter of apartments 
had these two mail access types (28 percent combined). Additionally, while observers were 
not able to determine the mail access type for about a quarter of apartments (23 percent), 
this outcome was very rare for single-unit and attached structures. Appendix D includes 
more information about subgroup variation in mail access type. 


The distribution of the mail access type among interview participants was similar to the 
distribution in the address observations, where a mail slot or mailbox attached to the home 
was the most prevalent mail access type (see table A.4.2 in appendix A). Most participants 
also mentioned the location of their mailbox during the interview. Those with mail slots or 
mailboxes attached to their home talked about the ease of having a mailbox close by. A few 
noted that the proximity enabled them to have a positive relationship with their mail carrier, 
making statements like, “Our mail carrier, he’s a cool guy. If he sees the door open, he'll ask me 
if I got anything, or I'll see him coming and give it to him.” (5703) Those with mailboxes at the 
end of their driveway, across the street, or at the end of the road also typically found their 
mailbox easy to access. In general, they did not talk about the location of their mailbox as 
being an issue or a huge impediment to checking their mail. However, for those with 
mailboxes not attached to the home, most did not talk about knowing their mail carrier. 


Consistent with the observation that apartment addresses were most likely to have a 
mailbox, slot, or room in multi-unit building, almost all participants living 
in apartments reported their building had metal or plastic mailboxes or slots with unit 
numbers on them, each requiring its own key. The exact location of these boxes varied, such 
as in the lobby or other common room of the apartment building, in an open-air communal 
space, or in a separate building (often requiring a key or passcode to enter). One participant 
described the layout at his apartment complex: “Each apartment has its own little mailbox, 
and every time the mailman comes, I go out and get my key, open my box, and get my mail.” 
(6221) A few participants in apartments noted that they felt the placement of their mailbox 
was inconvenient, with one saying, “Jt would be better if it was at the door.” (4495) 
Participants in apartments rarely mentioned their mail carrier and, unlike those with 
attached mailboxes, no one mentioned having a relationship with their mail carrier. 


4.1.3 Checking the Mail 
Frequency of checking the mail 


Over three-fourths of the interview participants noted how often they check the mail. The 
frequency ranged from twice a day to only once every six weeks, with most checking their 
mail at least once per day. Those who checked the mail every day tended to have a routine, 
such before or after work, after walking the dog, or when the kids returned home from 
school, noting behavior like, “When I come home from work, usually our mail carrier comes at 
the end of the day, so what happens [is] I'll check my mailbox [then].” (5195) 


Those who did not check the mail every day tended to engage with their mailbox only when 
they expected specific pieces, like bills, to be delivered. One participant noted, “/ get mail once 
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a month. I don’t check it every day. I don’t even know when it comes in every day. [I check when] 
my bills [come], usually my light bill comes on the 21% so I already got that down pat.” (5757) 
Many of these participants also noted that extreme demands on their time, due to work 
schedules or other responsibilities, could influence their level of engagement with the mail. 
One participant exemplified both reasons, saying, “Mail be sitting there for weeks at a time... 
because I be tired. After I drop [the kids] off from school, I sometimes fall asleep in my car. When 
I finally get up to come in the house, I go to bed. Sometimes if I know something is supposed to 
be coming, I be like, ‘Oh, go get the mail.’ But if not, I don't.” (7057) Additional reasons offered 
for not checking their mail very often included receiving too much junk mail, not regularly 
receiving important items, and not wanting to deal with the hassle of their mailbox being in 
another building. 


Of the participants who checked their mail once a week or less, few had children living with 
them (as compared to about half of all participants). The majority lived in an apartment in 
an urban area, and most were Black or Hispanic. Almost none of them ultimately responded 
to the NHES. There was no clear pattern in the type or location of their mailbox. 


Responsibility for checking the mail 


Responsibility for checking the mail appeared to vary by the number of adults living in the 
household. In households with two adults, most checked the mail every day, with about half 
having one person generally tasked with it. In the other half, the adults shared the 
responsibility. In these households, typically the adult who was present when the mail was 
delivered or who was the first to arrive home after work checked the mail. 


In households with more than two adults, mail checking responsibilities varied, and it was 
rarer for one person to be responsible for checking the mail. In some households, no one 
routinely checked the mail. In others, several people routinely did so. For example, when 
asked who picks up the mail, a participant who lived with five other adults said, “It’s whoever 
is home and whatever time the mail comes; like, whoever’s outside or whoever sees the mail 
coming here, is whoever grabs the mail.” (6081) 


4.1.4 Challenges with Mail Delivery 
About one-third of participants explicitly discussed challenges with mail delivery. Among 
participants who reported challenges, most noted that these issues happened frequently, 
from once a month up to several times a week; only a few indicated that these mishaps were 
rare. While mail challenges occurred across all structure types, they were mentioned more 
often by participants who lived in attached structures (e.g., duplexes) or apartments. 
Challenges tended to fall into one of five categories: 
e receiving mail addressed to a different physical address, 


e having their mail incorrectly delivered to their neighbors, 


e receiving mail addressed to landlords or former occupants, 
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e not receiving expected mail items, and 
e receiving mail that had been opened or tampered with. 


Participants who encountered these challenges offered several explanations. For those in 
apartment buildings or attached structures, some noted that it can be unclear which mailbox 
goes with which residence and that they sometimes receive mail intended for another unit. 
Interviewers also observed this lack of clarity while in the field, noting that, at some 
residences, apartment or unit numbers were missing from mailboxes. A few apartment 
dwellers also explained that mailboxes were often opened by people other than their owners 
and that mail theft was common, particularly for mailings that looked as if they could contain 
money. When explaining why he prefers his mail to go to his work address, one participant 
said that is “because [in] this area here, people are known to steal and to disrupt the flow. So, I 
would rather have everything coming to me, where I know it’s coming.” (5463) 


Participants living in single-unit structures who encountered mail delivery challenges 
frequently noted that they received mail with the incorrect house number or incorrect street 
name (typically with a similar sounding or nearby street name). One participant noted, “J 
thought I'd go to the post office to see why that’s happening. They leave my correspondence at 
my neighbor’s or they leave mine with him. I don’t think they take a good look at the number or 
the name.” (4131) 


Across all structure types, several participants noted that their mail carrier changed 
frequently or that the mail carrier chose not to deliver mail directly to their mail receptacle. 
A participant who lived in a duplex shared that the mail carrier would not deliver mail 
directly to his mailbox, which was at the top of a flight of stairs. Instead, his and his neighbor’s 
mail were left downstairs together in a pile (5703). 


One challenge noted by some renters was that the landlord or property owner used the 
sampled address as his or her permanent address even though he or she did not live onsite. 
As aresult, these renters frequently received the landlord’s or owner’s mail. Therefore, when 
mail was addressed to “the household” these renters often assumed it was intended for the 
owner. For example, during one interview, the interviewer noticed that a basket containing 
mail to be shipped to the homeowner had two of the NHES mailings in it (6535). Relatedly, 
some renters assumed that mail addressed to “current resident” was not meant for them and 
instead was intended for a previous tenant. 


Of those who explicitly talked about mail delivery challenges, the majority lived in an urban 
area. Most were Black or Hispanic, with most of the Hispanic participants living in 
households that predominantly spoke Spanish. There was no pattern in the type of mailbox 
that they had or their household structure type. 


4.1.5 Types of Mail Received 
According to participants, typical mail generally fell into five categories: 
e junk mail, 
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e bills, 
e official notifications (e.g., tax-, court-, bank-, or school-related items), 
e desired goods (e.g., magazines, checks, medicine), and 


e personal correspondence (e.g., letters or cards from friends or family, wedding 
invitations). 


The types of mail people received did not seem to vary by participant characteristics. More 
than half of participants routinely received substantial amounts of junk mail, with 
solicitations for credit cards or insurance offers being the most common. As one participant 
observed, “People spend a lot of money to mail stuff [that is] just to be thrown out right away.” 
(7329) Most participants generally thought of junk mail as an unavoidable nuisance 
although, as noted in earlier, a few felt inundated and burdened by the junk mail they 
received. As one explained, “I really, really don’t like all the mail I get. [Junk mail] is absolutely 
a waste of time and effort.” (5463) Participants tended to consider something to be “junk 
mail” if it was unexpected and irrelevant. A mailing that was unsolicited but addressed 
something of value or interest to the participant would typically not be labeled as junk. For 
example, some participants said that they found grocery store flyers valuable for 
ascertaining what items are on sale. Others felt that these circulars were advertisements and 
destined for the trash. 


About half of participants mentioned receiving bills through the mail. As one participant 
summarized, “Bills—electric, gas, cable, stuff like that.” (7329) The other half of participants 
stated that they now conduct their financial business online, including using online bill pay, 
and therefore no longer receive paper bills in the mail. One participant said, “We are online- 
driven: online bank, online bill pay, online everything. Everything we get in the mail is either a 
birthday or anniversary card or junk mail.” (7303) 


About one-quarter of participants said that they receive official notifications through the 
mail. The type of notifications received was diverse; some participants noted receiving mail 
from U.S. government offices, such as the Internal Revenue Service, Social Security 
Administration, or Immigration Services, while others mentioned receiving letters from 
their children’s schools. For people who did not check their mail often, expecting an official 
letter typically prompted them to visit their mailbox. One participant, who checked his mail 
less than once a week, explained what prompts him: “I just look for the ones that I know I need 
to look for, which is the U.S. Attorney’s Office, anything for my [refrigeration] license, and car 
insurance.” (4153) 


Finally, about 1 in 10 of participants said they receive desired goods, such as packages, 
magazines, or personal correspondence. One participant noted that her son is currently 
interested in the mail because of these types of deliveries: “We order stuff for him, so he’s 
expecting packages. He’s in the 10" grade, but he’s already getting some letters from colleges 
so he’s starting to look at that.” (6793) Personal correspondence tended to be holiday-related 
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cards or invitations, although one participant noted that she often receives handwritten 
letters in the mail from her son who is incarcerated. 


4.2 Processing Mail 


Almost all participants had a dedicated space to place mail once it entered their home, such 
as in a cabinet in the living room, on a small bookcase, in a basket, or loosely piled on the 
table. That said, several participants did not have an organized system for Keeping track of 
the items they received. For example, one participant kept a box of unopened mail in his 
closet while another kept stacks of unopened mail under a shopping bag. 


4.2.1 Approaches to Processing Mail 


Knowing more about how participants sort different mail items helps us understand the 
context in which the NHES survey mailings are handled once they arrive at sample members’ 
homes. Most participants, regardless of their demographic characteristics, tended to 
categorize mail into multiple groups that did not always fall neatly into “important” or “junk.” 
Often there was a third category—a gray zone—where it was not clear upon immediate 
inspection what a mailing contained. This gray-zone could include mailings that were 


e not urgent but might be important (like a communication from a child’s school); 
e confusing (unclear who the sender was or what the envelope contained); or 


e potentially interesting (but not necessarily interesting enough to override other 
extenuating factors, such as busyness). 


In this sense, many participants further separated “important mail” into “urgent” and “not 
urgent” categories. Urgent mail—like a bill needing to be paid—was prioritized and opened 
more quickly. Important mail that was not urgent, particularly if it was not expected, was 
kept but it was not always ultimately opened. Mail of indeterminate value—things that might 
or might not be important—also had the potential to be kept unopened before ultimately 
being discarded. This way of conceptualizing mail did not seem to vary across different 
subgroups. 


Participants who offered detailed descriptions of their approaches to processing mail 
outlined four discrete stages related to deciding: (1) when to sort mail; (2) how to sort mail; 
(3) whether to open junk or gray-zone mail; and (4) how long to keep junk or gray-zone mail. 
At each of the four stages, most participants used one of the following two approaches.!! 


When to sort mail 


In terms of deciding when to sort mail, about three in four participants sorted their mail 
immediately after checking the mailbox or as soon as they returned to their residence. In 


11 In total, about 80 of the 85 participants gave enough information for their approach to at least one of the four stages to be categorized. 
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these cases, there was no or very little time lapse between checking and sorting. As one 
participant explained, the minute she brings her mail inside, “I organize it into what it is. Is it 
from a business? Is it from a public agency? Is it a newspaper flyer? Is it the local magazine? Is 
it coupons? I organize it.” (5479) Another participant who lived in an apartment complex 
stated that she “usually goes through it [right] at the mail center” so she can toss unwanted 
mail immediately (4209). 


The remaining participants sorted their mail after participating in other activities (e.g., 
eating dinner, child care, sleeping), the following day, or even later. Here there was a 
substantial time lapse between checking and sorting. One participant described how she and 
her husband put each day’s mail into one stack and went through it, often together, when 
‘the pile gets too high.” (7583) Another participant said he did not need to sort immediately 
because he subscribed to the USPS Informed Delivery service, so he already knew what mail 
he was getting (4705).12 


How to sort mail 


During the sorting process, about three-fourths of participants examined their mail to 
identify which pieces they deemed the most important. They would pull out these pieces first 
and place them in a new pile or piles, separating them from the original mail stack. As one 
participant explained, “Anything that’s under my name, like any bill, anything I order or 
something, I look for it first.” (6081) 


Conversely, during the sorting process, about one-fourth of participants examined their mail 
to identify which mail pieces they deemed most likely to be junk. They would remove these 
pieces from the bundle first and place them into a separate pile. For example, when asked 
what she did first thing when looking at her mail, one participant exclaimed, “throw out junk 
mail.” (5449) 


Whether to open gray-zone or junk mail 


When deciding whether to open gray-zone mail—mail that is neither labeled as junk or very 
important—about half of the participants tended to default to opening a piece of mail 
whenever its relevancy or importance was in doubt. They would open an ambiguous mailing 
for several reasons including being “curious” about the content or wanting to check for 
personally identifiable information (PII) so they could dispose of it properly. One participant 
explained how even when something was likely junk, she would open it so she could use an 
ink roller to black out any personal information: “Not that we’re going to open up another 
credit card, or whatever these offers are, but I would still open it just to make sure. Like if there’s 
any personal information I need to block out before I just throw it away.” (4277) A few 
participants shared that not opening particular mailings in the past had caused them to miss 
important information or potentially lose something of value. For example, one participant 
shared that, about 20 years ago, she received an envelope that sat on her dresser for weeks 


12 This was the only participant who mentioned having USPS Informed Delivery service. This service allows households to see a digital 
preview of the mail that is scheduled to be delivered. 
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(5515). It turned out to be a substantial rebate check. Since that incident, she had opened 
every single envelope in her mail. 


The other half of participants tended not to open mail that was confusing or that they did not 
consider important. They gave several reasons for not opening this kind of mail. Most 
believed it was “inefficient” or “a time waster” to open unimportant mail. Others thought that 
anything truly important would not be sent by regular USPS mail (it would be sent via 
Priority mail or FedEx) or that truly important mail would be sent again if this piece went 
unanswered. Finally, some participants said that opening mail that was not immediately 
clear or urgent was just not a priority. In many of these cases, mail would stay unopened 
within the residence for an extended period of time—for example, because of health 
problems (4043) or procrastination (7329). 


Keeping and discarding junk or gray-zone mail 


Almost all participants kept urgent mail in a specific location. That said, the locations ranged 
from very specific—like filed in an office desk or on a shelf near the front door—to general— 
like the kitchen or the bedroom. Participants differed on how they kept important and 
unimportant—or gray-zone—mail. While a few kept all mail together, most separated urgent 
mail from all other mail. For example, one participant kept important mail in a file box and 
urgent mail thumbtacked to the bottom ofa wall calendar (4209). Another participant stored 
important and unimportant mail separately, with a household folder labeled “unnecessary” 
for unimportant mail (7321). 


Most participants discussed how to dispose of unwanted or no longer needed mail. Among 
those participants, many said that they tossed it in the trash or recycled it. Some said that 
they cut or shredded all or most mail before disposing of it. A minority of these participants 
shared that they took additional measures such as burning or bleaching their mail so it would 
be completely destroyed. One participant described her multi-step process: 


If it has my name on it, I'll rip it up. But if it’s something like a credit card offer, 
I'll either shred it in the shredder machine, or I will take the scissors, and the little 
fake credit cards that come in those offers, I'll cut them up. Because sometimes 
they'll say my name on them. So, I'll actually cut those up and cut them to where 
you're not able to put [them] back together. Things like that I will destroy. 
There’s been times when I'll collect a bunch of stuff. I'll keep it together and then 
I'll take them to one of the community events. I work in the medical field. 
Obviously, HIPAA [Health Insurance Portability and Accountability Act] and all 
of those things are our first priority so I’m familiar with [companies that destroy 
documents]. So we find out about the community events a lot, and we'll take our 
own stuff: Sometimes, they charge you. And sometimes, it’s free. So, we'll do that. 
And my dad has even taken it and burned it because he lives in the country. So, 
he’s taken bags of my stuff and will burn it. (4399) 


Of those who shred or tear their mail, about half had a bachelor’s degree or higher. 
Conversely, of those who took more extreme measures (burning or bleaching), no one hada 
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college degree. Those in the more extreme category also tended to be female and most did 
not have children living in the home. 


Lastly, when deciding to dispose of junk and gray-zone mail, just over half of the participants 
kept mail in this category in the house longer than one day. Many of these participants also 
put this kind of mail into, as one participant phrased it, “a set-aside pile” intending to follow 
through on reading or responding to it but ultimately never returning to it (6175). Pointing 
to a table, one participant said, “I could show you that pile right there and you'd be like, ‘Wow, 
this guy, he hasn’t really opened up his stuff, but he’s kept up with it.’”” (6393) 


In contrast, the rest of the participants discarded mail quickly. In these cases, they discarded 
mail they deemed to be junk or gray-zone mail within the day and sometimes immediately 
after determining its lack of relevancy. A participant whose mail access at the apartment 
complex was in a separate building explained the process: “I take a bag with me when I go 
over there because that sucker is jam-packed full of just advertisements really. I throw [them] 
away at the place. They have a little trash can there that’s full of coupons that somebody didn’t 
want.” (4298) 


Subgroup variation in approaches to processing mail 


There were differences in mail processing behaviors between participants who ultimately 
responded to the NHES and those who never responded. About two-thirds of late 
respondents tended to live in households that did not have a break between mail retrieval 
and mail sorting, compared to just over half of participants who did not ultimately respond 
to the NHES. About three-fifths of late respondents also tended not to open gray-zone or junk 
mail and disposed of mail quickly, compared to about a third of participants who did not 
ultimately respond. 13 


There were also some variations across demographic subgroups. Participants who took a 
break between checking and sorting tended to be White, Spanish-speaking, or Hispanic. 
Those who pulled out gray-zone or junk mail first tended not to have children in the 
household. Participants who usually opted to open gray-zone or junk mail more often lived 
in households with three or more adults and, than participants who generally did not open 
unsolicited mail. Compared to participants who kept non-urgent mail in their house longer 
than one day, a higher proportion of participants who disposed of their mail quickly were 
White. 


4.2.2 Mail Characteristics Driving Sorting and Opening Decisions 


Participants were asked to review and sort a prepared mail bundle as if it was their regular 
mail delivery—and to verbalize their reactions to each piece of mail as they sorted the mail 
bundle according to their usual routine. The bundle contained an assortment of mailings, 


13 While this finding may seem counterintuitive, it is possible that these participants did not consider the NHES mailings to fall into a 
“gray zone”; as noted later in this document, most participants said they would engage with the NHES screener envelope during the 
example mail activity. The chapter 5 (“survey attitudes and experiences”) findings shed more light on participants’ perceptions of the 
NHES mailings. 
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with each containing a different combination of features, such as the personalization of the 
recipient name, the sender name, the envelope size and color, the type of stamp, and other 
embellishments (e.g., “open immediately”). Several of the mailings were addressed to “Jaime 
Smith,” and participants were instructed to imagine that this was their name. Exhibit 4.1 
provides an overview of the mailings used in this activity; copies of the mailings are included 
in appendix G. The remainder of section 4.2 summarizes the comments that participants 
made during this example mail activity. 


Sorting and opening mail 


Many participants talked about sorting mail by priority, with urgent mailings receiving 
immediate attention, junk mail relegated to the trash pile, and gray-zone mailing rising or 
falling in importance based on envelope features. Unless they tended to open all or most of 
their mail, participants tended to scan the envelope for clues that would help them determine 
a course of action. In these cases, participants typically first reviewed the name of the 
recipient or the sender. For example, one participant said she would automatically shred any 
mailing where she did not recognize the sender (5691). However, no single feature 
consistently garnered the same reaction from all participants. Nor did an identical feature 
across mailings always prompt the same reaction from a single participant. 


Instead, participants tended to make judgments based on the totality or combination of 
envelope features. For example, if a mailing was addressed with the full name of the recipient 
but the envelope also had features that suggested it was a solicitation (such as “open 
immediately” or “0% interest”), many would not engage, even if they had previously stated 
that including their full name was the overriding factor in determining engagement. 
Additionally, many participants noted that they were wary of the marketing tactics 
employed to encourage people to open mail. One participant relayed this experience: “/ 
received a letter and from the outside it was like a personal letter with my name written in ink, 
and I was like, ‘This is something personal.’ And I open it, and it was trash. And then I started 
thinking about it. I tell my husband, ‘Why [do] they take that whole time to put trash inside?’ 
And he said, ‘Because they want to grab your attention.” (4271) 


For those participants who either did not open all their mail or did not open their mail 
immediately after retrieving it, other environmental contexts—such as work schedule, 
busyness, and health condition—also influenced their decision whether or not to open a 
piece of mail. As one participant said, “If it says [my name] and spell[s] my name incorrectly, 
then it’s just those little things that kind of trip you down on the legitimacy. And sometimes 
when I’m more time-pressed, I’m less discriminating. I'll rely upon those small factors and throw 
it out. It’s worth it more for me to minimize my time than it is to review every aspect of my 
mail.” (7705) 
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Mailing Size / shape Envelope color Special notes Recipient name Special features Font type Postage 
U.S. Department of Letter-size White Official Business Member of Anytown | Windowed Standard Prepaid 
Commerce envelope (NHES envelope Census logo household Barcode 
initial screener package) 
Personal notecard envelope | Small White None Jaime Smith None Handwritten Forever stamp 
National Center for Public Large Priority Mail | White Priority mail Member of Anytown | Stick-on address Standard Priority label 
Opinion Research envelope envelope Tracked, insured household label 
Barcode 
National Council for Letter-size Light blue None Smith household Stick-on address Standard Stamp 
Education envelope envelope label 
Dentist appointment Postcard Not applicable Appointment Jaime Smith None Standard Prepaid 
reminder card reminder 
Utility company envelope Letter-size White Open Immediately | Current resident Windowed Standard Prepaid 
envelope 
USPS National Customer Letter-size Manila Address Service Smith household or None All caps Prepaid 
Support Center envelope envelope Requested current resident 
Insurance envelope Letter-size White Business Reply Jaime Smith None Italicized / Metered 
envelope Mail cursive 
Candidate forum flyer Half-page glossy Not applicable None Jaime Smith None Standard Metered 
flyer 
Quality Bankers envelope Letter-size White Save Now / 0% Jaime Smith None Standard Prepaid 
envelope Interest 
Box store circular Square multi-page N/A N/A N/A Color photos on N/A N/A 
booklet exterior; printed on 
newspaper-weight 
paper 
Mail order catalog Rectangular multi- | N/A N/A N/A Color photos on N/A N/A 
page booklet exterior; printed on 
glossy, heavy-weight 
paper 
Note: Because most of the materials did not have very much text on them other than the mailing address and sender name, most of the materials were presented in English only. The three exceptions were the NHES 


initial screener package, the dentist card, and the candidate forum flyer, for which Spanish versions were shown to Spanish-speaking participants. 
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Recipient name 


In general, participants said that mail that felt more personalized, with traits like the 
recipient’s first and last names, was more likely to be opened. In this sense, personalization 
increased the chance that they would open a piece of mail. Furthermore, some participants 
said they would only open mail that had their first and last names on it, with one noting, “If 
it’s not addressed to me ... I would never open it.” (4705) Conversely, a few participants were 
not swayed by personalized mailings. One participant “would question why it has my name 
because I've never heard of [the sender]” (7503), while other participants noted that, because 
companies can buy lists of people’s names, envelope personalization may still not be 
personal. Similarly, generic recipient names such as “occupant” or “current resident” were 
almost always deemed to be junk mail. Generally, only participants who said they routinely 
open all or most mail engaged with these mailings. 


When the mailing was addressed to the household, responses were mixed. In the example 
mail activity, when the recipient was “Smith Household,” some participants believed that 
their last name paired with “household” warranted engagement with the envelope. Others 
believed that using the term “household” implied that the mailing was not urgent or did not 
require individual action. As one participant explained, being addressed to “household” 
“means it’s not to any specific person in my house. It’s to whoever opens it, so then I know it’s 
not got anything vital in there that I’ve got to know about or use or do or act upon.” (4167) 


Mail that was not specifically addressed to an individual but to the household (such as “Smith 
Household”) appeared to be particularly problematic for multi-adult and Spanish-speaking 
households. In multi-adult households, this type of mail generally did not have a clear way 
to be sorted. Some participants said that these envelopes would by default go to the head of 
household. Other participants said that they would look for clues on the envelope to gauge 
relevancy. For example, if something on the envelope said “education,” the person sorting 
the mail would give that piece to the person most closely tied to education, such as a college 
student or a parent of grade school children. In Spanish-speaking households, several 
participants said that it was unclear what “household” (which was written in English) meant 
on the National Council for Education envelope, with one wondering if it meant it must 
pertain to the physical house itself (4541). 


There were some differences across other subgroups when looking at which participants 
tended not to open mail that was not specifically addressed to an individual. Of those 
households that routinely discarded these types of mailings, most did not have a college 
degree and many did not have children in the household. Very few of these households 
ultimately responded to the NHES. Additionally, about half of the households that spoke a 
language other than English tended not to engage with generically addressed mailings or 
those addressed to the household. 


Sender name 


Previous communication or relationships with the sender, such as one’s insurance or utility 
company, often nudged participants into opening a mailing. For example, during the example 
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mail activity, participants who noted that they had insurance through the company listed in 
the return address of the envelope said they would likely open the mailing from that 
company, whereas participants who said they used a different insurance company believed 
it to be an advertisement and would ultimately discard it. As one participant said in 
summary, “I know! don’t have that company or anything to do with it [so] I don’t pay attention 
to it.” (4131) In that sense, knowing the sender shed light on what the mailing likely 
contained, and a participant’s desire—or lack thereof—for that communication influenced 
engagement. 


In some cases, not recognizing a sender meant the mailing was destined to be thrown away. 
A participant explained, “IfI didn’t know the [return] address ... I wouldn’t even open it.” (5103) 
However, in cases where participants were not familiar with the sender, if the return address 
referred to a topic or concept that was important to them, they leaned toward engaging with 
the mailing. For example, one participant said she would open the mailing from the National 
Council for Education because it may have to do with her children who were in school (5703). 
In that sense, if a sender seemed relevant to the participant, any unfamiliarity with it became 
less important. Similarly, some participants noted that receiving repeated mailings from the 
same sender, even if they did not recognize the return address at first, would prompt them 
to open the later mailings because they assumed that multiple attempts to reach them must 
mean that the correspondence was important. 


Not all participants said that the sender directly influenced their decision to engage with a 
mailing. As noted earlier, some participants routinely engaged with all mailings. Other 
participants said they rarely looked at the sender or that it did not make a difference. 


Other envelope features 


Having a non-white envelope sometimes caught people’s attention, as they noticed “the 
bright color.” (4277) Other times the participant said it made no difference, with one 
participant noting: “I’m not drawn by color. I’m more drawn by text. The color doesn’t have 
information, the text has information.” (5195) In general, when a letter-size colored envelope 
caught people’s attention, it was interpreted as a sign of junk mail and an “attention-getter.” 
(7303) One participant noted that yellow can mean important, so “that’s why a lot of other 
businesses sometimes catch you off guard and they put something in a yellow envelope. It'll 
have a stamp on it, and it says important, but it is usually promotional [junk].” (6535) Likewise, 
the phrase “open immediately” was generally interpreted as a sign of junk mail. Participants 
tended to equate this phrase with a credit card offer or other type of solicitation. 


To almost all participants, an envelope larger or smaller than standard letter size suggested 
that the sender took extra time or money to send the mailing. Several also noted that they 
associated larger 8.5-x-11 mailers with important mail since the envelope likely contained 
documents. As one noted, “Whatever’s in there, it means it’s some kind of document you need 
to keep.” (5703) However, a few participants observed that using this metric was becoming 
less reliable because companies were increasingly putting junk mail in larger envelopes. As 
one participant explained, “Because I work in business, I know strategies specifically to get 
people to open their mail and sending things Priority is 100% a strategy.” (5203) 
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Most participants did not discuss the postage type. Of those who did, they interpreted real 
“Forever” postage stamps to bea sign that personal care was involved in sending the mailing. 
For example, one participant explained that the real stamp means “it looks like it was 
packaged with care. They ... [weren’t just] not sending it out to the masses.” (4277) While 
metered postage was generally interpreted as a sign of a mass mailing, participants said that 
it would not dissuade them from engaging with a mailing. Like the stamp, few participants 
noted the presence of “address service requested.” Those who did had mixed reactions. Some 
thought it made a mailing more “legitimate” or “official.” Others believed the language made 
it less personal or signified that the mailing was “garbage.” 


4.2.3. Engagement with and Reactions to Mail Pieces from Example Mail Activity 


The next section continues with a discussion of the reasons participants gave for engaging 
or not engaging with each mailing from the example mail activity. Figure 4.2 shows the 
percentage of participants who said they would engage with each of the mailings (also see 
table A.4.3 in appendix A). Participants who noted that they would open, read, or keep a 
given item were coded as “would engage.” Among those not already coded as “would 
engage,” participants who stated they would not open, read, or keep the mailing were coded 
as “would not engage.” Any remaining participants were coded as “unclear.” Because of the 
more structured nature of this activity, we report the percentage of interview participants 
that said they would engage with each mailing, and the subgroup analyses conducted for this 
section include more participant characteristics than considered in earlier parts of this 
report (e.g., age, gender, household income) (see table A4.4 in appendix A). Because of the 
small number of interview participants, we focused on patterns and did not conduct 
statistical testing.'4 


U.S. Department of Commerce envelope (NHES initial screener package) 


Ninety-two of the participants stated they would engage with this mailing. In general, 
participants assumed this mailing was about the Census or somehow related to the U.S 
government. Many noted that, although the mailing was addressed to the household and not 
a specific individual, their belief that this mailing was government-related overrode any 
concerns about lack of individualization. As one participant summarized when looking at the 
mailing, “Obviously [this mailing] would catch my eye, so I would open that right away. Cause 
it looks important ... cause it’s from the government.” (4277) 


Those who would not engage explained that they would not open the envelope either 
because it was from the government or was not addressed directly to them. For example, one 
participant noted that the Census Bureau was “not my favorite people in the world” (5611), 
while another noted, “/t’s not addressed to me. It’s from the U.S. Department of Commerce and 
Statistics. So, it’s not something that I need to address urgently or else it would be specifically 
addressed to me.” (5479) 


14 Subgroup differences in engagement are noted below when the subgroup was composed of six or more participants and the 
percentage reporting they would engage was at least 10 percentage points higher or lower than the overall percentage of engagement 
with a given mailing. 


69 


Chapter 4. Receiving and Processing Mail 


Figure 4.2. Percentage distribution of interview participants’ self-reported engagement with example mailings, by 
example mailing: 2019 
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SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Surveys Program (NHES), 2019. 
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For participants who would engage, when asked to inspect the mailing, about three in ten 
said there were aspects of the envelope that also made them reluctant to actually open it. For 
some, the sender information caused confusion or suggested caution. For example, several 
participants believed that the Department of Commerce was related to money, and as one 
participant explained, this made it seem that the government was “trying to get data on my 
household incomes, budgets or spending habits.” (4399) The term “penalty for private use” 
was wotrisome and off-putting to a few participants, with one wondering if she would incur 
the $300 penalty if she did not open the mailing (4043). To some, the information on the 
envelope, when carefully studied, invited more questions, or as one participant summarized, 
“The more I read it, the more confusing it might be.” (7149) Other participants pointed to the 
Census logo and noted that the current year (2019) was not a Census year, so they wondered 
if the letter was a scam. 


Knowing how participants process mail, this finding suggests that engaging initially does not 
always equate to ongoing engagement. It appears that participants interpreted the envelope 
at first glance as being urgent, likely because they believed it was from the government. 
However, either upon further inspection of the envelope or reading the materials inside, for 
some participants, the envelope was reclassified as gray-zone and/or the urgency was 
downgraded. 


Personal notecard 


The combination of the notecard size, the handwritten text, the real stamp, and the personal 
name led 89 percent of the participants to say they would engage with the personal notecard. 
Participants said it seemed “personal” and could be an “invitation.” One participant said that 
the combination of features made him think that “this is from someone I care about.” (4639) 


For those who said they would not engage, they noted that they did not recognize the 
sender’s name or address, which made them suspicious or uninterested in opening it. One 
participant said, “I don’t know if I would open it or not. It would depend on if I know who the 
person [who sent it].” (5203) Additionally, some participants explained that they had seen 
solicitations using a handwriting type font, so they could not automatically assume it was 
personal, with one noting that “they’re getting more crafty. They make it seem like 
handwritten, and you look close and say, ‘This is actually printed to look handwritten.” (7149) 


Participants who refused to share their household income during the interview were less 
likely to say they would engage with this mailing. In contrast, all participants in the following 
subgroups said they would engage: those who were 18 to 24 years old; 65 and older; 
currently enrolled in school; or with a household income of $30,001 to $60,000. 


National Center for Public Opinion Research envelope 


Participants tended to make decisions about this mailing quickly, likely because it was in a 
Priority envelope, which was both larger than a standard envelope and labeled as “Priority.” 
Eighty-eight percent of participants said they would engage with this mailing, noting that its 
Priority status meant that the sender spent additional money on delivery, which indicated 
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that its contents were important. Several participants believed it looked “more official” and 
others thought that materials sent Priority suggested that the sender “has a deadline to meet” 
(5195) or that recipients were purposively selected to receive the envelope as opposed to a 
mass mailing. As one participant observed, “Someone puts something in a sealed Priority 
envelope for a reason. ... Very rarely do you get sent something [Priority] that was randomly 
sent.” (6535) Only a few participants said that seeing “research” in the sender’s name was 
meaningful to them and helped encourage them to open it. Those who said they would not 
engage assumed the mailing was for someone else because they were not expecting it or they 
believed it to be junk mail since it was not addressed using their first and last name. 


Several subgroups were slightly less likely to say they would engage with this mailing: 
participants for whom the interview was conducted in Spanish; those living in households 
where Spanish was the language most often spoken at home; participants who were 18 to 
24 years old; and those living in households with internet access via their phone or tablet 
only. All late respondents said they would likely engage with this mailing, as did all 
participants age 65 or older. 


National Council for Education envelope 


Roughly 80 percent of participants said that they would engage with this mailing. These 
participants tended to point to the sender “National Council for Education” as important. For 
some, this made it seem that the mailing may be related to their child’s school. As one 
participant observed, “This is kind of educational, right? Yes, I always keep everything from 
school for [my daughter] because she’s in the process of going to university.” (4017) For others, 
education in general was a salient topic to them or someone they knew. When asked why she 
would open this mailing, a participant explained, “because it says education on it and since 
education is first and foremost with me, I would open this up and see what the contents of this 
is.” (5463) 


Participants who said they would not engage offered one of three reasons. Some pointed out 
that the envelope was not directly addressed to them. Others did not believe it was relevant 
to their household because they did not have any school-age children. A few people believed 
it could be a solicitation, likely about student loans. 


Participants whose household income was over $100,000 were less likely to say they would 
engage with this mailing. In addition, participants who were 45 to 54 years old, had some 
college but no bachelor’s degree, or were the only adult in the household were also less likely 
to engage. In contrast, participants who were 18 to 24 years old, Hispanic, currently enrolled 
in school, or had household income between $30,001 and $60,000 were more likely to report 
that they would engage with it. 


Dentist appointment reminder card 


Seventy-two percent of the participants said they would engage with the reminder card. 
They interpreted the card as being from their actual dentist, with many saying they would 
physically keep the card (often on the refrigerator) to act as a reminder. One participant 
shared, “Obviously that would be something that’s [from] my dentist; I would take that as 
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legitimate.” (7055) However, several participants who would engage said they would 
ultimately not keep the card because they receive appointment reminders via e-mail or text 
or would have already recorded the appointment on a virtual calendar. Those who said they 
would not engage at any level misunderstood the nature of the card and believed it was an 
advertisement. 


Participants in certain subgroups were more likely to say they would engage with the card: 
those who were ages 18-24 or 35-44; who were Hispanic; who were living in households 
where Spanish was the language most often spoken at home; who had a high school degree 
or less; and who were currently enrolled in school. In contrast, participants who were ages 
25-34, White, or who had some college but no bachelor’s degree were less likely to say they 
would engage. 


Utility company envelope 


Roughly 60 percent of the participants said they would engage with the envelope from a 
regional utility company.!5 These participants believed that it could be legitimate 
correspondence since they were customers. As one participant exclaimed, “Oh boy, time for 
the light bill!” (5409) Not everyone who had services through that utility company believed 
it was official business about their account. A few participants pointed out that it was not 
addressed to a particular individual, so it must be a solicitation or scam. Others for whom 
this was their utility company were ambivalent about engaging with the mailing because 
they participated in online bill pay and were unclear what kind of correspondence they 
would be receiving from this company. Participants who said they would not engage stated 
that it was either not their utility company or that their utilities were covered by another 
entity, like the apartment complex. One participant in this group explained, “I got a rate I’m 
comfortable with. Very seldom do I look at these.” (7465) 


Participants ages 18-24 were among the most likely to say they would engage with this 
mailing. Additionally, participants in the following subgroups were more likely than average 
to say they would engage with it: those who were ages 35-44; who were Hispanic; who had 
a graduate degree, who were currently enrolled in school; and who had three or more adults 
in the household. In contrast, participants ages 55-64 or who had household incomes higher 
than $100,000 were less likely to say they would engage. Those who were age 65 or older 
were the least likely to say they would engage. 


National Customer Support Center envelope 


Fifty-six percent of the participants said they would engage with this mailing, generally 
pointing to the return address with “United States Postal Service” that led them to believe 
that this mailing was official or legitimate. As one participant said, “There might be some 
information in there, like changes or something going on.” (7503) Those who said they would 
not engage with the mailing believed it to be junk because they did not recognize the return 
address or because it was addressed to “current resident.” One participant succinctly spoke 


15 The name of the utility company varied by site to reflect the actual name of a local utility company. 
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to both reasons, “I wouldn’t open it because I don’t know what it is about, and it doesn’t have 
my name on it.” (6237) 


Overall, there was considerable subgroup variation around engagement with this mailing. 
All or most participants who were ages 18-24, were currently enrolled in school, or had 
household incomes between $30,001 and $60,000 said they would engage with the mailing. 
Participants who were ages 25 to 34, were Hispanic or were living in a household with three 
or more adults also were more likely than average to say they would engage. Conversely, 
participants who were age 55 or older, had household incomes between $60,001 and 
$100,000, whose interview was conducted in Spanish, or who declined to share their 
household income were less likely to engage with the mailing. 


Insurance envelope 


Fifty-five percent of the participants said they would engage with the insurance mailing. Like 
the utility company mailing, participants who said they would engage did so because they 
felt it was relevant to them: the sender was their own insurance company or they might want 
to take advantage of a lower insurance rate. Several participants suggested that it must be 
an “insurance bill,” because the recipient was a specific person (“Jamie Smith”) and not 
“household.” Others thought it was a solicitation since they did not have Progressive, but that 
it could still be relevant because it could get them a lower quote on automobile insurance. 


Participants who said that they would not engage stated that it was not relevant to them 
because it was not their insurance company or they did not own a car. As one participant 
stated, “I have insurance and I’m happy with it, so there’s no reason to [open it]. And I wouldn’t 
buy from somebody sending me a generic mailer like that. I would ask people who they use and 
if they are happy with them and that’s how I would purchase.” (4373) 


Most participants who were ages 18-24, were currently enrolled in school, or refused to give 
their income said they would engage. Likewise, many participants who were Hispanic or had 
a high school education or less said they would engage. In contrast, participants ages 55-64, 
with some college or a bachelor’s degree, or with a household income of $60,000 or more, 
were less likely to say they would engage with the mailing. 


Candidate forum flyer 


About 40 percent of the participants said they would engage with the flyer, explaining that 
they were interested in local issues that were impacted by politics. One participant explained 
he might attend a forum to meet other people and “if it’s right where I’m living at, I should go 
check it out.” (7717) Those who said they would not engage were either not interested in 
politics in general or did not find community events like forums useful (even if they were 
vested in political issues). For example, one participant said that it was not necessary to 
attend a candidate forum because she hears about local issues on the news (5409). 


About half of the participants who were ages 35-44, were Hispanic, had household incomes 
between $30,001 and $60,000, or refused to give their household income, said they would 
engage with the flyer. In contrast, participants who were ages 18-24 or 55-64, had some 
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college but no bachelor’s degree, or had household incomes of less than $30,000 or more 
than $100,000 were less likely to say they would engage. 


Mail order catalog/Box store circular 


About 40 percent also said they would engage with the box store circular or the mail order 
catalog. Participants said they would engage with these catalogs and circulars if they were 
interested in shopping for something in particular with the company or wanted something 
to browse during their downtime. A few participants who said they would not engage said 
that they preferred to browse advertisements online or look at sale items in person at brick- 
and-mortar stores, while others considered them advertisements or junk mail. 


For the box store circular, some subgroups had higher levels of engagement: about half of 
the participants whose interview was conducted in Spanish, who were ages 3544, who were 
Hispanic, who had a high school degree or less, who were not employed for pay, who had a 
household income less than $30,000, and who had home internet access ona phone or tablet 
only, said they would engage. In contrast, participants who were 65 years or older or were 
currently enrolled in school were less likely to say they would engage with the circular. There 
was even less reported engagement among participants with a bachelor’s degree or who had 
a household income greater than $100,000. 


For the mail order catalog, about half of the participants in the following subgroups said they 
would engage: late NHES respondents; participants whose interview was conducted in 
Spanish; those ages 35-44; those who were Hispanic; those who had completed some college 
but no bachelor’s degree; those who were not employed; and those with household incomes 
between $30,001 and $60,000. In contrast, participants who were White, were male, or had 
household incomes higher than $60,000 were less likely to say they would engage. 


Quality Bankers envelope 


Thirty-one percent of participants said they would engage with the Quality Bankers 
envelope. Participants made decisions about this mailing quickly. Those who said they would 
not engage said it was clear from the “0% interest” text and return address that it was either 
a credit card or bank offer, and, therefore, it was clearly junk mail. Notably, most people who 
said they would engage also believed it was a credit card or bank solicitation but said they 
wanted to check whether the contents had any personally identifiable information. For 
example, one participant explained, “/’m not interested in it, but I'll open it. Because what I 
tend to do is, I'll cut out my name and then I'll shred it into pieces.” (5195) Very few who said 
they would open it would do so because they wanted to read and assess the information 
within the mailing (for example, because they desired a new credit card). Even though this 
mailing had the recipient’s full name (“Jamie Smith”), participants did not believe it was 
personal. Interestingly, several participants stated while sorting through the example mail 
bundle that this mailing was not directly addressed to them, suggesting that other features 
of the envelope such as “0% interest” cemented in their minds that this mailing was junk 
before they examined the recipient’s name. 
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Participants who were 18 to 24 years old, who had a high school degree or less or lived with 
three or more adults said they would engage were the most likely to engage. In contrast, 
participants who were 25 to 34 years old, had some college or a bachelor’s degree, or were 
the only adult in the household were less likely to say they would engage. 
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Chapter 5. Experiences with and Opinions About Surveys 


This chapter presents findings on participants’ experiences with and perspectives on 
surveys, both in general and for the NHES more specifically. By understanding the factors 
that influence survey response decisions, we can better understand how to improve 
response to future NHES administrations. The goals of this chapter are threefold: (1) provide 
insight into participants’ prior experiences with and attitudes toward surveys in general; (2) 
identify the key factors and influences that shape participants’ decisions on whether to 
participate in surveys in general and the NHES specifically; and (3) determine whether 
participants remember receiving the NHES screener mailings—and, if so, the extent to which 
they engaged with them. 


Section 5.1 covers participants’ responses about their experiences with and views on 
surveys in general. Section 5.2 describes the results of an activity in which participants were 
shown the NHES:2019 screener materials and asked to provide feedback about them. 


5.1 Experiences with and Opinions About Surveys in General 


In this section, we first describe participants’ responses about their prior experiences with 
and attitudes toward different types of surveys. Throughout this section, we highlight 
subgroup differences in experiences or opinions using the same key characteristics of 
interest that were outlined in chapter 2. 


5.1.1 Prior Experience with Surveys 


About nine-tenths of participants reported that they had completed one or more surveys in 
the past. Discussion of these prior survey experiences centered around two dimensions: (1) 
survey type (based on a combination of the survey topic and sponsor) and (2) survey 
response mode. 


Survey type 


Participants reported experiences with several types of surveys. Feedback surveys and 
government surveys were mentioned most often. A smaller number of participants had 
completed other types of surveys. These are discussed in greater detail below, presented in 
order from most to least commonly mentioned. 


Feedback surveys. Three-fifths of participants had taken part in feedback surveys. One- 
third of participants had completed a customer satisfaction survey in which they were asked 
to provide feedback to a commercial business on various consumer products or services. For 
example, one participant shared, “I’ve been doing a lot of [surveys]... Just all types of product 
surveys and stuff: Manufacturer surveys. I guess people want to know what brands are [in] and 
stuff like that.” (5265) Many of these participants shared that they were asked to fill out these 
feedback surveys immediately after they received a product or service. 
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In addition, a little over a fifth of participants shared that they had participated in surveys 
where they provided feedback to social institutions, such as hospitals or schools. One 
participant said, “It’s like when you're hospitalized, and they ask you how they treated you. 
‘What do you think [the hospital] should change?’ ... Things like that are what I've filled out.” 
(5429) Some participants who were students (or had been students in the past) shared that 
they had filled out course or instructor evaluation surveys. 


For both types of feedback surveys, unless they were compulsory, some participants were 
motivated to participate because of negative experiences, while others were motivated by 
positive experiences. For example, one participant explained, “So, in those [positive] 
instances, I do deem it as [valuable]. [If a] person did a good job, I do tend to leave an all-right 
survey... If they did a [bad] job, I usually don’t even bother.” (5203) 


Government surveys. About half of all participants mentioned that they had responded to 
a government sponsored survey in the past. Most participants’ experiences with government 
surveys came from taking part in the Decennial Census. Chapter 3 provides a more detailed 
discussion of participants’ attitudes towards the Decennial Census. In addition, a few 
participants shared that they had taken part in issue-specific surveys sponsored by their 
local government. For example, one participant shared, “I guess I participated in a survey of 
housing, and it was at home and they asked me, ‘How long have [you] been living here?’ I said, 
‘Oh, I've been here like fifty, almost fifty years.”” (5515) 


As described in chapter 3, although some participants reacted positively to the idea of the 
government collecting personal information through surveys, many others reacted 
negatively because they believed the government already had access to this information, 
were skeptical of how the government would use this information, or were scared of being 
scammed or targeted by the government. Typically, participants who had positive views of 
government institutions in general or who conveyed a strong sense of agency to effect 
change in their community tended to share that they had participated in government- 
sponsored surveys in the past. In contrast, participants who distrusted the government or 
who felt that their participation in government sponsored surveys did not make any 
meaningful difference to society tended to share that they typically ignored government- 
sponsored surveys (see “Contributing to the greater good” and “Makes no difference” in 
section 5.1.2 for more details). 


Employer surveys. Less than one-tenth of participants shared that their experience of 
surveys came through their workplace. Typically, these surveys took the form of employee 
engagement surveys designed to get a pulse on employees’ morale, concerns, and 
expectations. For example, as one participant said, “They'll always send us something for work. 
We just had our employee engagement survey. So just the other day, I took one of those online.” 
(7015) 


Other work-related surveys were designed to elicit employees’ input on how to improve 


things. One participant who worked at a university shared, “At [university], they send 
confidential surveys for all the faculty members to talk about diversity and inclusion. And they 
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were open-ended questions, and they reassured us that [it was] anonymous. So, they’re gauging 
the climate on campus on diversity and inclusion.” (5195) 


Social science surveys. Less than one-tenth of participants had participated in surveys as 
part of social science research. For example, one participant had responded to a survey as 
part of a larger research project on education barriers affecting the local Puerto Rican 
community (5195). Another had responded to a survey that was part of a research study on 
social media usage (5479). These participants tended to have a bachelor’s degree or more as 
their highest level of education or to have research experience themselves (via their 
education or employment). These participants also tended to express a strong 
understanding of the role of surveys in research and their value to society. As one reflected, 
“Data and evidence... surveys serve a purpose to collect data. And then, just my clarifying 
question is, how is the data going to be used? ... Is that survey going to turn into a plan of action 
for the community that you serve?” (5195) Based on their understanding of the role of 
surveys, these participants tended to believe that their participation in social science 
research surveys made a positive contribution to knowledge. 


Political opinion and polling surveys. Very few participants reported that they had 
completed political opinion or polling surveys. The surveys they had completed tended to 
relate to national rather than local politics and were often described in politically partisan 
terms (e.g., Republicans, Democrats). Participants who shared that they had participated in 
these types of surveys also tended to convey that they were politically active or engaged. One 
participant shared that she was motivated to participate in surveys to convey her political 
preferences (4025). Notably, nearly all the participants who shared that they had 
participated in political opinion or polling surveys were White. One White participant shared 
that he was inclined to participate in political surveys because he grew up in a cultural 
context in which politics were often discussed: “Yeah, I get a lot [of political surveys] because... 
It gets around that I’m into politics and you can’t hide it. And especially my mom was into it. I 
mean, we talked politics like people eat breakfast, lunch, and dinner.” (5463) 


Market research surveys. Finally, even fewer participants reported that they had 
responded to market research surveys that asked them to provide opinions on products or 
services with which they were unfamiliar. For example, one participant said, “We’ve done 
surveys in the past before, through some companies, but they’re talking to the kids about soda 
or about toys.” (6793) 


Survey response mode 


In addition to participants’ experiences with different types of surveys, participants 
discussed their experiences with a variety of response modes (again presented from most 
commonly to least commonly mentioned). 


Online. Around half of all participants indicated that they had completed a survey online. 


For example, one participant said, “I don’t usually go for [surveys], unless it’s online and I get 
money. Actually, yeah, okay I’ve done a few surveys for money online.” (4639) Younger 
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participants (between the ages of 18 and 34) reported participating in online surveys more 
often than did participants in older age groups. 


Paper. About a tenth of participants indicated that they had completed a paper 
questionnaire. Some participants indicated that these paper surveys had been mailed to their 
household. While online surveys were the most commonly reported response mode among 
participants overall, Hispanic and Spanish-speaking participants’ experiences tended to be 
with non-web-based surveys, such as paper or phone surveys. 


Phone. Less than a tenth of participants indicated that they had completed a survey over the 
phone. Most participants’ experiences with phone surveys were for customer feedback 
surveys conducted at the end of a phone-based interaction with a vendor or service provider. 


In-Person. Only a couple of participants indicated that they had completed an in-person 
survey. For example, one participant said, “/ believe [I took the survey] in-person the first time 
and then the follow-up questions or the follow-up survey was through the mail.” (5103) 


5.1.2 General Attitudes Toward Survey Participation 


Among the participants who shared that they had taken part in a survey in the past, about a 
third shared positive views about participating in surveys, a third expressed negative views, 
and the other third were neutral. The participants who had more consistently positive views 
about surveys tended to be White or have children living in the household. In addition, late 
respondents to the NHES tended to have more positive attitudes toward surveys overall 
compared to final nonrespondents. Participants who had more consistently negative views 
about surveys tended to be Black or not have children living in the household. Participants 
who generally had neutral views about surveys tended to be female or lived alone. 


Positive attitudes toward survey participation 


Participants who expressed generally positive views about surveys gave the following 
reasons for why they tended to participate: improving products and services, contributing 
to the greater good, and personal gain (again, presented in order from most to least 
commonly mentioned). 


Improving products and services. About half of the participants who had positive attitudes 
toward surveys felt that these surveys helped improve the products and services they use 
regularly. As one participant said, “Sometimes, occasionally, if I have a few minutes, Ill [do an 
online feedback survey] like, ‘How was your hotel stay?’ and ‘How was your [treatment] ata car 
dealership?’ I'll do that because it may influence their treatment of other people.” (7149) 
Participants without a bachelor’s degree tended to speak about survey participation 
improving products or services that they interact with directly, while participants with a 
bachelor’s degree or more as their highest level of education tended to describe the benefits 
of surveys in terms of how they affect broader society (see “Contributing to the greater good” 
below). 
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Contributing to the greater good. Some of the participants who had positive views toward 
surveys expressed the opinion that they participated in surveys to benefit society in general. 
One participant said, “If I knew...something good was to happen from just hearing about other 
people doing a survey...then maybe I [would] consider it... need more information as to... what 
happens after the survey’s been [taken].” (7057) These views demonstrated these 
participants’ belief that they could contribute to tangible change in their communities and 
society at large through their participation in government and research-based surveys. 


These views were most commonly mentioned in relation to government surveys like the 
Census; participants tended to believe that the information they provided on these types of 
surveys helped the government make positive changes. Late NHES respondents and 
participants with a bachelor’s degree or more as their highest level of education were 
especially likely to note that government surveys had the potential to contribute to the 
greater good. For example, one late NHES respondent stated, “I think we try to participate in 
[government surveys] because we feel like it’s helping out, especially when it comes from [the] 
Census Bureau or an education bureau. So, you might as well participate to try to help the 
community.” (7303) 


In contrast to more abstract notions of helping society, participants without a bachelor’s 
degree tended to emphasize more specific, localized benefits of participating in surveys. For 
instance, one participant whose highest level of education was high school said, “/fyou want 
my honest opinion, anything to help the community, yeah, I’m down for it. I'll take a survey. 
Potholes and all that they need to [be fixed] here in [this state].” (7401) 


A few participants emphasized that surveys help the government stay informed and make 
better policies. One said, “I think [the Census] is very important because it gauges a shifting 
demographic of communities... Gathering this data is going to benefit most demographics.” 
(5195) Others also believed that their participation in government surveys helped determine 
how the local and federal government, as well as other public institutions, allocated 
resources. 


Similarly, participants who had participated in research-based surveys shared that they 
were motivated to do so because they wanted to help advance knowledge in an important 
area affecting society. One participant explained: “/t was a study when I was pregnant... It was 
about pregnant women that were having hypertension or something, and I was part of a group 
that had severe hypertension during my pregnancy, so I was like, ‘Oh, well this might be 
something that I would want to do.’ If I can help... with stuff like that, then I don’t mind doing 
that.” (5103) 


Personal gain. In contrast to participants who spoke about the benefits of survey 
participation in terms of how it contributed to the greater good, a few participants described 
how they personally benefited from their participation in surveys. Some of these participants 
shared that their main motivation in participating in surveys was to receive money or other 
rewards (see “Incentives” in section 5.1.3 for more details). Others shared that their 
participation in surveys had taught them more about certain issues. As the following 
participant described: “/Surveys] can give me a little knowledge on what’s going on around, 
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like I said, with education, the voting, the [political] candidates. But that depends on what the 
survey consists of.” (4495) These participants indicated that they would only be willing to 
participate in a survey in the future if it was clear how they would benefit personally. 


Negative attitudes toward survey participation 


As noted above, about a third of participants shared generally negative views of surveys. 
Typical reasons included having survey fatigue, feeling that surveys do not make a difference, 
having concerns about privacy and information security, and being too busy or having too 
little time (again, in order from most to least commonly mentioned). 


Survey fatigue. About half of the participants who expressed negative views about surveys 
shared that they found survey solicitations to be too frequent and aggressive, particularly 
online. As one participant said, “I’ve had these things pop up, survey things pop up online and 
they tell you, ‘You win this if you answer the survey.’ And then after you answer the survey, they 
have you answer another one. Then another one. There’s like a hundred. So, I just don’t answer 
those anymore.” (5449) 


Moreover, some participants with negative views about surveys shared that they were 
turned off by prior experiences with surveys that promised prizes or rewards (e.g., cash 
prizes) but turned out to be advertisements. In particular, Black participants shared that 
they had this experience more often than did participants from other racial/ethnic 
backgrounds. For example, one Black participant said, “I’m not really big on surveys because 
I've participated in far too many of them that made [false] promises... So, I don’t participate in 
surveys anymore.” (6393) These repeated negative experiences caused participants to lose 
trust in surveys overall. 


Makes no difference. As discussed in chapter 3, some participants with negative attitudes 
toward surveys shared that they did not participate in surveys because they did not believe 
that surveys could lead to any benefits or meaningful changes—for themselves or for society 
more generally. This perspective was especially prominent among Spanish-speaking and 
Hispanic participants. For example, one Spanish-speaking, Hispanic participant explained 
why she chose not to participate in government surveys like the Census: “Well, for me, I don’t 
think that it would serve me. It serves the government... But for me, I don’t get anything from 
it.” (4711) In contrast to participants who felt that their participation in surveys like the 
Census helped benefit society in some way, these participants tended to convey a lack of 
agency to influence change in general. Participants who lacked a general sense of agency 
tended to be unmotivated to participate in surveys, viewing them as a waste of time. 


Concerns about privacy and information security. The general concerns about privacy 
noted in chapter 3 also were discussed specifically in relation to survey participation. A few 
participants who expressed negative attitudes shared that they were not inclined to 
participate in surveys because of concerns that their personal information could be stolen or 
used in unwanted ways. One participant noted, “Sometimes, they say it’s a survey, but it’s not 
really a survey. Sometimes, it’s somebody trying to steal your information.” (5757) 
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Other participants believed that some survey solicitations were veiled attempts by 
marketers and other entities to access contact information (e.g., e-mail address, phone 
number) for future solicitations, a perception that discouraged participants from sharing 
their personal information through surveys. For example, one participant shared why he had 
misgivings about participating in both marketing and government surveys: “I’d have my 
suspicions on both. One, I’m going to be marketed to, and two, with the government I always 
worry... I’m not sure what’s going to happen with the information. I’m not certain how much 
I’m going to like divulging certain things.” (7055) As alluded to by this participant and 
described in greater detail in chapter 3, several participants believed that their information 
was already being shared or sold by the government and private entities without their 
consent or knowledge. 


Concerns over privacy and information security as they relate to surveys diverged 
depending on participants’ education levels. Those that did not have a bachelor’s degree 
mentioned concerns about the security of their personal information when participating in 
surveys less often than participants with a bachelor’s degree or more. Moreover, among 
those who expressed concern over information security, the focus of this concern tended to 
differ based on their highest level of education. For example, participants with a bachelor’s 
degree or more tended to frame their concerns about informational security around their 
distrust of government institutions. As one participant with a graduate degree said, “/I] don’t 
make it a point to complete government surveys ... I don’t trust them. [I] don’t know what [the] 
end result is.” (4365) In comparison, participants that did not have a bachelor’s degree tended 
to frame their concerns about information security around their negative past experiences 
with surveys that turned out to be scams. 


Hispanic and Spanish-speaking participants’ reactions toward sharing their personal 
information ranged from positive to apathetic or fearful. Some of the participants who 
expressed a fear of filling out government surveys shared that they felt vulnerable due to 
their immigration status. One Spanish-speaking Hispanic participant said that “no, we didn’t 
fill [government surveys] out... because we were afraid. We thought that by filling it out, well, 
since it was from the government...one is afraid of, well, yes, that they could come and kick us 
out that way.” (4541). In comparison, Black participants tended to be less likely to mention 
concerns about the security of their personal information on surveys. 


Too busy, too little time. A few of the participants who had negative attitudes toward 
surveys noted that they were too busy in their daily lives to take part in surveys. These 
sentiments also were shared by many participants who conveyed neutral attitudes toward 
surveys. Participants who worked long hours or multiple jobs shared that taking time to do 
a survey was not feasible given more pressing economic priorities. A few participants who 
had children living in the household shared that it was not feasible for them to find time to 
do surveys because there were so many demands on their time. For example, one participant 
who lived with young children said: “I just don’t have time to sit and do surveys though ... I 
would probably have something on my stove burning as I was going through this pile ... 
Honestly, it’s so hard to get a few minutes uninterrupted that there’s no way.” (5103) Others 
shared that their time was constrained because of specific problems they were going 
through, such as medical issues. Under difficult circumstances, participants shared that it 
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was unlikely that they would expend the time or the energy to complete a survey. Chapter 3 
provides a more detailed discussion of this busyness and the time constraints that 
participants faced. 


In addition, several participants shared that the reason for their lack of participation had less 
to do with having time than with the timing of the request. These participants conveyed that 
they were more likely to respond to survey requests that caught them at the right time—in 
other words, when it was convenient for them. As one participant explained, “/I’ll participate] 
if I’m on the computer at that time. Yeah, only if I’m on at that time. I’m not going to look for 
them.” (6175) 


5.1.3. Survey-Specific Influences on Survey Participation 


In addition to the general attitudes discussed above, participants also identified several 
survey-specific factors that influenced their decisions about participating. These factors 
included the topic, the sponsor, whether incentives were offered, the mode of response, the 
length, and the language of the materials (again, presented from most to least commonly 
mentioned). 


Survey topic 


Participants generally indicated they were more likely to complete a survey if they felt 
personally, politically, or morally aligned with the topic of the survey. For example, one 
participant shared: “I’m selfish so anything that’s going to [help] my family first... I think in the 
past year, I got a food survey. And that was important because they were talking about [the] 
quality of food and quality of vegetables and fruits. So, that was important because it kind of 
affected the way we eat.” (4399) 


Similarly, participants with no interest in the topic of the survey tended to ignore survey 
solicitations. For example, one participant explained why he ignored a polling survey 
request: “Polling stuff, I’m not interested in at all... I just don’t subscribe to the games of this 
country. I mean the political games. I don’t subscribe to any of that.” (4289) 


Just over a third of the participants from households with children mentioned that they 
would be motivated to participate in a survey if the topic focused specifically on education, 
compared to about a tenth of participants from households without children. One late NHES 
respondent who had three children in school described why he was highly motivated to 
participate in education-related surveys: “I feel a sense of wanting to help education because 
we're at the forefront of our kids getting it. Education, it’s important to us, right? Just like 
voting. We very much focus on those topics. We always vote, but those are very key to our 
interests.” (7303) Similarly, just under a quarter of the participants from households without 
children indicated that they would purposefully ignore education-related surveys because 
they felt that education-related surveys were not relevant to them. 
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Survey sponsor 


Participants generally indicated that they were more likely to participate in a survey if they 
knew and trusted the sponsor of the survey. The following exchange with one late NHES 
respondent who explained her thought process when making this decision provides an 
example: 


Interviewer: What's the difference to you [between] government surveys and 
the surveys for market research? ... Would you be more inclined to answer one 
over the other? Do you have a distinction between them? 


Participant: Well, the distinction, if it’s market research, again, it will depend if 
it’s for a nonprofit... I think that it depends on...the mission of it. 


Interviewer: What kind of survey would you not answer? 


Participant: For me, [what] would just give me pause is [the] legitimacy of the 
agency ... That’s a huge thing. And I tell my students, ‘Google is my friend’... And 
I’m going to type it, I'll ask questions, I'll call, and I'll verify ... But ifit doesn’t feel 
right, then I won't do it at all. (5195) 


As discussed above in the section “Contributing to the greater good,” participants who 
expressed higher levels of trust in government tended to express more willingness to 
complete government-sponsored surveys. In contrast, participants who generally did not 
trust the government tended to avoid government-sponsored surveys. 


Incentives 


Some participants indicated that they would be more motivated to respond to a survey if an 
incentive (e.g., cash, gift card) was offered. This view was especially prominent among 
participants who did not have a bachelor’s degree. Statements like the following from a 
participant, whose highest level of education was a high school degree were common: “I did 
one last week when I went to [a fast-food restaurant]. It took five minutes, so that’s why I was 
like, “[Heck], yeah! I'll take a $50 gift card for five minutes.” (4289) Some of these participants 
shared that they specifically sought out surveys to earn money through cash incentives. 


Other participants expressed the opposite view about incentives. As discussed further below 
in the context of the $5 incentive sent as part of the NHES:2019 screener materials, these 
participants shared that they reacted to cash incentives with suspicion, expressing fear that 
free money seemed too good to be true and that they were somehow being scammed. 


Mode of survey response 


Participants generally indicated that they were more likely to complete a survey if it was 
presented in an accessible, user-friendly, and secure format that made it easy for them to 
participate at their convenience. For these reasons, many participants indicated a preference 
to complete surveys online, followed by paper and then by phone. Participants shared a 
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range of opinions on how each of these response modes shaped their decision to participate 
in surveys. 


Web-based surveys. Participants who preferred web surveys felt that web-based surveys 
were faster and more convenient to complete, provided they felt the web instrument was 
generally secure. In particular, younger participants (between ages 18 and 34) shared that 
they tended to prefer taking surveys online over other response modes. For example, a 
participant in his 30s shared the following: “/Web-based surveys are] quicker and easier to do 
without having to receive and send back via mail and to just do it when it pops up on the screen 
as a prompt or in an e-mail from a website.” (5479) 


While some participants expressed a positive view of web-based surveys because of their 
convenience, participants who were less familiar with the Internet or who had been victims 
of online scams viewed web-based surveys with suspicion. For example, participants who 
had a natural distrust of computers or sharing information online were more wary of 
participating in online surveys. Participants age 55 or older tended to share that they did not 
trust or were too unfamiliar with computers/internet to share their information via web- 
based surveys. Even for participants who were comfortable with web-based technologies, 
some conveyed that they did not like completing surveys online. One participant explained, 
“I don’t even mess with [online surveys] no more because when you do that mess, next thing you 
know, you start getting all these calls from people... And they just keep calling.” (5703) As noted 
in the “Survey fatigue” section above, persistent and aggressive online solicitations 
dissuaded some participants (particularly Black participants) from participating in surveys 
overall. 


Paper surveys. Participants who preferred paper surveys felt that paper surveys were more 
tangible and thus more secure. Participants age 55 or older were more likely to mention 
preferring paper surveys than were younger participants. For example, one participant in 
her 60s said, “Me personally, I would do [the paper survey]. Over[all], I’m not real good with 
computers...My kids and my husband would have no problem with the computer.” (7583) As 
this participant described, most participants who preferred paper surveys expressed a 
corresponding discomfort with computers or sharing information online. 


On the other hand, participants who expressed negative views toward paper surveys shared 
that they took too much time or were too complicated to fill in and send back. When asked 
whether he preferred web-based or paper surveys, one participant said, “I’d rather do it 
online. That way you don’t have to send nothing in. You get it right there, give your feedback.” 
(7465) As this participant described, negative views about paper surveys were typically 
expressed relative to the convenience of web-based surveys. 


Phone surveys. Most participants who expressed an opinion about phone surveys had 
negative views. Typically, these participants felt that they took too long and made them more 
vulnerable to potential scams. When asked if he would be willing to participate in a phone 
survey, one participant responded, “No. Because I just feel there’s too much scamming going 
on. Now, I will say had I not seen you and you personally, I would not have had an interview. So, 
ifyou’d called me on the phone, I would've said, ‘You sound very nice, but I’ve got to go.”” (7627) 
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The very few participants who expressed a willingness to participate in phone surveys felt 
that this response mode was faster and offered them an opportunity to communicate their 
views verbally. Older participants (age 55 or older) tended to be more open to participating 
in phone surveys than were younger participants. 


Survey length 


Not surprisingly, participants generally stated that they preferred shorter surveys over 
longer ones. Consistent with the finding above that some participants do not participate in 
surveys because they have little or no time to do so, participants generally preferred shorter 
surveys because they took less time to complete. Participants did not report a specific survey 
length that would be considered acceptable or too long. 


Language of survey materials 


A few participants who were native Spanish speakers noted that difficulty reading English 
was a barrier that prevented them from participating in some surveys. As described in more 
detail below in relation to the bilingual materials sent as part of the NHES screener materials, 
Hispanic and primarily Spanish-speaking participants indicated that receiving Spanish 
materials would encourage them to respond to the survey. Some participants noted, 
however, that they tend to have low expectations for Spanish materials in general because 
the translation from English tends to be poor quality. 


5.2 Engagement with and Reactions to NHES:2019 Screener Materials 


The remainder of this chapter discusses participants’ recollections of and views on the 
NHES:2019 screener materials. Their responses were gathered during a screener materials 
review activity that was included in each interview. The objectives of this activity were to 
better understand (1) how far participants tended to progress in the response process (e.g., 
Did they remember the mailings? Did they open them? What did they do with them after 
opening them?); and (2) the factors and mailing features that shaped participants’ reactions 
to the survey request. 


5.2.1 NHES:2019 Screener Materials Review Activity Methods 


Following discussion of the interview domains and the example mail sorting exercise (see 
chapter 4),1© participants were then shown and asked to provide feedback on the following 
NHES:2019 screener mailings, which had been sent to their address during the main 
NHES:2019 data collection (see appendix F for copies of the materials):17 


16 In addition, six individuals declined to do a full interview but completed a shorter, 5-minute interview. These interviews focused on 
determining whether the participant recalled getting any of the mailings and, if so, their reasons for (not) responding. The findings from 
these interviews are embedded in this analysis. 

17 Some sample members also received one or more advance mailings prior to the initial screener package. Due to interview length 
limitations, the review activity did not include those mailings. 
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1. The initial screener package. This mailing, which was sent in a letter-size envelope, 
included a letter that invited sample members to complete the survey online and 
included personalized login credentials for doing so. It also included a $5 cash 
incentive and a Commonly Asked Questions (CAQ) enclosure. 


2. The pressure-sealed envelope. This mailing consisted of a pressure-sealed form, 
the interior of which reminded sample members of the request to complete the 
survey online and included personalized login credentials for doing so. 


3. The second screener package. This mailing, which was sent in a letter-size 
envelope, included a letter reminding sample members of the request to complete the 
survey online and included personalized login credentials for doing so. It also 
included a CAQ enclosure. 


4. The third screener package. This package, which was sent in a large envelope, 
included a reminder cover letter, a paper copy of the screener questionnaire, a CAQ 
enclosure, and a postage-paid return envelope. 


5. The fourth screener package. This package, which was sent in a large envelope, 
included a reminder cover letter, a paper copy of the screener questionnaire, a CAQ 
enclosure, and a postage-paid return envelope. 


For the purposes of this activity, participants were shown NHES:2019 screener mailings 
that were addressed to “Member of AnyTown Household.” Interviewers asked participants 
to pretend that they lived in AnyTown, which was a mocked-up name for a city.!8 The 
return address for all mailings was the U.S. Department of Commerce, Economics and 
Statistics Administration, U.S. Census Bureau. Some features of the mailings varied between 
NHES sample members; during the mailing activity, participants were shown the type of 
mailings that their household were sent during the NHES:2019 administration. 19 


All mailings were presented in unsealed envelopes to allow participants to comment on the 
exterior of the envelopes as well as the mailing contents. Interviewers asked participants 
whether they remembered the mailings. If so, they asked what participants had done with 
the mailings after receiving them and what aspects of the mailings had driven participants 
to make these decisions. They also asked participants what they liked or did not like about 
both the exterior of the mailings and the cover letters included in them. At the end of the 
activity, they also asked participants to provide detailed feedback on the paper 


18 The real NHES:2019 screener mailings that were sent to sample members through the mail were addressed to “Member of [City] 
Household” (which was prefilled with the actual city in which the household was located). 

19 As part of an experiment embedded in NHES:2019, some sample members were randomly assigned to receive the second screener 
package via FedEx, while others were randomly assigned to receive it as a USPS First Class mailing. Sample members then received the 
opposite mailing type for the fourth screener package (e.g., sample members who received the second screener package via FedEx 
received the fourth screener package via USPS First Class Mail). About 50 of the interview participants were in the FedEx second/First 
Class fourth condition and about 35 of them were in the First Class second/FedEx fourth condition. In addition, while some sample 
members received English-only materials, those who were anticipated to have a higher likelihood of preferring Spanish materials 
received bilingual English/Spanish materials. About 20 of the interview participants received English-only materials for all mailings, 
while about 65 received bilingual materials for one or more mailings. 
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questionnaire, asking about things such as the cover images, the formatting, and the screener 
items themselves. 


5.2.2 Engagement with the NHES Materials 


During the NHES materials review activity, interviewers sought to determine whether 
participants remembered the mailings, whether they opened them, and what they did with 
the mailings after they opened them. For each mailing, we reviewed participants’ comments 
during the activity and determined where their engagement with that mailing fell within the 
engagement flowchart shown in exhibit 5.1. 


e First, we classified each participant as having either remembered or not remembered 
each mailing. When it was not clear whether the participant remembered a specific 
mailing, we coded these cases as “unclear whether remembered.” 


e Among participants who remembered a mailing, we repeated this process to classify 
participants as having opened or not opened the mailing. 


e Finally, among participants who had opened a mailing, we again repeated this process 
to classify participants as either having completed the survey, having saved the 
mailing to address later, or having rejected or discarded the mailing.2° 


Drawing on these mailing-specific recall rates, we calculated overall rates across all five 
mailings. For example, we created an overall recall rate that indicated whether participants 
recalled at least one of the five screener mailings and an overall opening rate that indicated 
whether participants opened at least one of the mailings. By developing these categories, we 
were able to establish more precisely when participants tended to drop out of the 
engagement process, which, in turn, helped narrow down the drivers of survey nonresponse. 
Based on these classifications, the remainder of this section explores participants’ recall of 
and engagement with the mailings in greater detail. Because of the more structured nature 
of this activity, we report the percentage of interview participants that reported each 
outcome of interest (e.g., the percentage of participants that recalled at least one mailing). 


In the subsequent section, we summarize participants’ reactions to the mailings, discussing 
the aspects of the NHES:2019 design and mailings that participants said drove them to 
remember the mailings or engage with them in a particular way. Both sections also highlight 
places where there was variation between subgroups in their level of engagement with the 
mailings. Due to the more structured nature of this activity, in addition to the priority 
subgroups discussed in earlier chapters, we included additional participant characteristics, 
such as participants’ age, gender, household income, educational enrollment, and 
employment status, and their observed structure type and mail access type. 


20 While all other classifications were based on participant reports during the interviews, survey completion status generally was based 
on the final outcome assigned to participants’ address for the NHES:2019 collection, regardless of whether or not participants stated that 
they had completed the survey. 
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Exhibit 5.1. NHES screener mailings engagement flowchart 


Completed survey 


(oyey=tat-xomaat-)ilial:4 Saved for later 


Remembered 1D)(ol ato) mace) aay o) (aa) 
mailing survey 


1D)(ol aXe) me) e-Vamant-li ilar Did not save for later 


Did not remember 
mailing 


Remembering mailings 


The first outcome in the engagement process was whether participants remembered 
receiving the mailings. As shown in figure 5.1, 76 percent remembered receiving at least one 
of the mailings (see also table A.5.1 in appendix A).2! These participants recalled a range of 
different aspects of the mailings. For example, one shared that he remembered that the initial 
screener package was sent by the government: “This is government. I remember when I first 
got it—I remember I read through the whole thing, but then I just put it down.” (5441) Another 
participant shared that he remembered receiving the pressure-sealed envelope: “IJ remember 
this and the envelope... Now, it’s just like [a] follow-up.” (5479) 


Participants remembered the individual mailings at different rates. 


e They most often remembered the initial screener package; 60 percent of them did so. 
As one participant shared, “The one for this survey. Yeah. It had the five dollars in it... I 
only found the one. It was this one [the initial screener package]. The regular envelope 
one.” (4131) 


e Participants were next most likely to remember the third screener package; 48 
percent of them did so. One participant shared that she remembered the third 
screener package because she had completed and returned the paper survey that was 
included inside: “This was the one that came. Of course, this one came in Spanish and 
English. This was the one that I sent [back with the completed survey].” (5305) 


e Participants were somewhat less likely to remember the other mailings, with 35 to 
41 percent noting that they remembered the pressure-sealed envelope, second 
screener package, or fourth screener package. Participants were no more likely to 
remember the FedEx mailer version of the second and fourth screener packages than 
they were to remember the First Class mail version of these packages. 


21 Half of the participants who completed the short interview also recalled having received at least one mailing. 
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Figure 5.1. Percentage distribution of mailing recall, by screener mailing: 2019 


Percent 
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80 
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At least one mailing Initial screener Pressure-sealed § Second screener Third screener Fourth screener FedEx package First Class Mail 
package envelope package package package package 


Screener mailing 


m@ Recalls m Does not recall m Unclear 


NOTE: Percentages are based on interview participants’ self-reports during an activity where they were shown each of the NHES screener mailings. It is possible that additional participants received the mailings but 
either did not recall doing so or did not mention it during the interview. It is also possible that a different household member would have recalled the mailing. Rounded number of eligible interview participants is 85. 
Details may not sum to totals due to rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Surveys Program (NHES), 2019. 
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As shown in table 5.1, the extent of the differences between subgroups in terms of 
remembering at least one of the mailings varied considerably by the characteristic being 
considered. For instance, there was practically no difference between men and women but 
considerable variation by age; only a third of participants ages 18 to 24 remembered at least 
one mailing, while most participants in the other age brackets did so. For additional 
subgroup findings, see appendix D. 


Table 5.1. Percentage of interview participants that reported remembering at 
least one NHES:2019 screener mailing, by selected characteristics: 2019 


Percentage of 
interview participants 
Number of interview that remembered at 


Selected characteristics participants least one mailing 
Total 85 76.5 
Nonresponse study site 
Texas 25 73.9 
Connecticut 20 85.0 
California 15 78.6 
Ohio 30 71.4 
Nonresponse study interview language 
English only 70 76.1 
Spanish only 10 88.9 
Mix of English and Spanish 5 60.0 
NHES:2019 final screener response status 
Responded 15 86.7 
Did not respond 70 74.3 
Observed structure type! 
Single-unit 50 72.9 
Attached 10 100.0 
Apartment 25 73.1 
Observed mail access type? 
Mail slot or mailbox attached to the home 40 76.3 
Mailbox at the end of the driveway, across the street, or at 
the end of the road 25 73.9 
Mailbox, slot, or room in multi-unit building 5 85.7 
No mailbox or slot in view 10 60.0 
Age? 
18-24 10 33.3 
25-34 15 86.7 
35-44 20 83.3 
45-54 20 89.5 
55-64 15 71.4 
65 and older 10 75.0 
Refused $ $ 


See notes at end of table. 
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Table 5.1. Percentage of interview participants that reported remembering at 
least one NHES:2019 screener mailing, by selected characteristics: 
2019—Continued 


Percentage of 
interview participants 


Number of interview that remembered 
Selected characteristics participants _at least one mailing 
Gender? 
Male 35 76.5 
Female 50 76.0 
Refused $ $ 
Race/ethnicity? 
White, non-Hispanic 25 72.0 
Black, non-Hispanic 25 81.5 
Hispanic 25 78.3 
Other race, non-Hispanic 5 66.7 
Refused $ $ 
Education? 
High school or less 40 73.7 
Some college, but no bachelor's degree 25 76.0 
Bachelor's degree 15 73.3 
Graduate degree 5 100.0 
Enrollment status? 
Enrolled 10 80.0 
Not enrolled 75 77.0 
Refused $ $ 
Employment status? 
Employed for pay 60 81.0 
Not employed for pay 25 66.7 
Household income? 
$30,000 or less 25 69.6 
$30,001-$60,000 20 86.4 
$60,001-$100,000 15 714 
$100,001 or higher 15 78.6 
Refused 10 75.0 
Language spoken most often by adults in household? 
English 65 76.1 
Spanish 15 76.9 
English/Spanish equally + $ 
Other $ $ 


See notes at end of table. 
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Table 5.1. Percentage of interview participants that reported remembering at 
least one NHES:2019 screener mailing, by selected characteristics: 
2019—Continued 


Percentage of 
interview participants 


Number of interview that remembered 

Selected characteristics participants _at least one mailing 
Home internet access? 

No access + + 

Phone/tablet access only 20 77.8 

Computer 65 75.0 

Refused $ $ 
Child in household? 

Yes 40 73.8 

No 45 79.1 
Number of adults in household? 

1 adult 25 88.0 

2 adults 40 75.6 

3 or more adults 20 63.2 


Reporting standards not met. There are too few cases for a reliable estimate. 

1Attached structures include duplexes, townhouses, and rowhouses. Apartments include low-, medium-, and high-rise apartments. Structure type was not 
able to be determined for a very small number of cases, and these cases have been excluded from this analysis. 

2A few cases where mail was received in more than one way were categorized under "mail slot or mailbox attached to the home" since all the ways mail 
was received fit that category. Mail access type was not able to be determined for a very small number of cases, and these cases have been excluded from 
this analysis. 

3These characteristics are based on self-reports provided by interview participants. 

NOTE: Percentages represent the percentage of interview participants that reported having remembered at least one NHES screener mailing during an 
activity where they were shown the NHES:2019 screener mailings. In the small number of cases where more than one household member participated in 
the interview, recall is based on the primary interview participant's recall of the mailings. Sample sizes are rounded to the nearest 5. Percentages are 
rounded to one decimal place but have not been changed to reflect sample size rounding. Details may not sum to totals due to rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Surveys Program (NHES), 2019. 


For the quarter of participants who did not remember receiving any of the NHES mailings, 
there are several potential drivers of this lack of recall. The mailings may not have reached 
the address, someone else in the household may have been the one to check the mail on the 
day the mailings came, or the mailings may not have resonated with participants. The 
subgroup findings in table 5.1 provide some support for each of these hypotheses. For 
example, households with more adults were less likely to remember at least one mailing; as 
noted in chapter 4, households with three or more adults were more likely to share mail 
processing responsibilities among several household members. It is plausible that someone 
else in these households would have remembered the mailings. In addition, addresses for 
which observers could not locate mailboxes were less likely to remember at least one 
mailing; these addresses might also have been more likely to experience mail delivery issues. 
However, we cannot definitively say which of these led individual participants not to recall 
the mailings. 
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Opening mailings 


Overall, 61 percent of participants reported opening at least one of the mailings.24 Among 
the 76 percent of participants who remembered at least one of the mailings, 80 percent also 
reported opening at least one.23 The second rate disentangles opening from remembering 
and provides a better sense of the rate at which participants consciously chose whether or 
not to open the mailings. 


Some participants said they remembered the contents of the mailings, which implied that 
they opened them when they first received them as part of the national study. For example, 
after opening the initial screener package during the interview, one participant shared, 
“These I think [are] the same letters that I received ... I put [the incentive] to the side... I want 
to say [it’s familiar] because some of it was sent in Spanish.” (4399) 


As shown in table 5.2, there was some subgroup variation in the rate at which participants 
who recalled at least one mailing also reported opening one of them. For example, 
participants with children living in the home reported opening at least one mailing at a 
higher rate than did participants who did not have children living in the home. Similarly, 
participants who lived in two-adult households were more likely than other participants to 
report opening at least one mailing. For additional subgroup findings, see appendix D. 


Table 5.2. Percentage of interview participants that opened at least one 
NHES:2019 screener mailing among those that remembered at least one 
screener mailing, by selected characteristics: 2019 
Number of interview 


participants that Percentage that 
remembered at least opened at least 
Selected characteristics one screener mailing one mailing 
Total 65 80.0 
Nonresponse study site 
Texas 15 70.6 
Connecticut 15 82.4 
California 10 90.9 
Ohio 20 80.0 
Nonresponse study interview language 
English only 55 77.8 
Spanish only 10 87.5 
Mix of English and Spanish + t 


See notes at end of table. 


2 Due to the semistructured nature of the interviews, there was variation across interviews in the extent to which participants discussed whether 
they had opened every single mailing. In addition, because opening is dependent on recall, the number of participants included in the opening 
analysis is by necessity smaller than the number available for the remembering analysis. Therefore, for the analysis of open rates, we focus only 
on the rate at which participants opened at least one mailing, and we do not report mailing-specific open rates. 

3 At first glance, this finding may appear somewhat at odds with the finding from the mail activity in which nearly all the participants stated that 
they would engage with the NHES initial screener package envelope. However, in that context, “engagement” with a mailing also included 
reading the exterior of the envelope, which is a lower bar than opening it. In addition, the mail activity reported on hypothetical actions, whereas 
much of the NHES materials review activity focused on participants’ actual behaviors when they had received the mailings; individuals’ 
intentions may not always end up being in line with their actions. 
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Table 5.2. Percentage of interview participants that opened at least one 
NHES:2019 screener mailing among those that remembered at least one 
screener mailing, by selected characteristics: 2019—Continued 


Selected characteristics 
NHES:2019 final screener response status 
Responded 
Did not respond 
Observed structure type! 
Single-unit 
Attached 
Apartment 
Observed mail access type? 
Mail slot or mailbox attached to the home 


Mailbox at the end of the driveway, across the street, or at 
the end of the road 


Mailbox, slot, or room in multi-unit building 
No mailbox or slot in view 
Age? 
18-24 
25-34 
35-44 
45-54 
55-64 
65 and older 
Refused 
Gender? 
Male 
Female 
Refused 
Race/ethnicity? 
White, non-Hispanic 
Black, non-Hispanic 
Hispanic 
Other race, non-Hispanic 
Refused 
Education? 
High school or less 
Some college, but no bachelor's degree 
Bachelor's degree 
Graduate degree 


Number of interview 
participants that 
remembered at least 


one screener mailing 


15 
50 


35 
10 
20 


30 


15 


Percentage that 
opened at least 
one mailing 


100.0 
75.0 


80.0 
100.0 
73.7 


See notes at end of table. 
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Table 5.2. Percentage of interview participants that opened at least one 
NHES:2019 screener mailing among those that remembered at least one 
screener mailing, by selected characteristics: 2019—Continued 


Number of interview 
participants that 
remembered at least 


Selected characteristics one screener mailing 
Enrollment status? 
Enrolled 10 
Not enrolled 55 
Refused $ 
Employment status? 
Employed for pay 45 
Not employed for pay 20 
Household income? 
$30,000 or less 15 
$30,001-$60,000 20 
$60,001-$100,000 10 
$100,001 or higher 10 
Refused 10 
Language spoken most often by adults in household? 
English 50 
Spanish 10 
English/Spanish equally + 
Other $ 
Home internet access? 
No access + 
Phone/tablet access only 15 
Computer 5 
Refused $ 
Child in household? 
Yes 30 
No 35 
Number of adults in household? 
1 adult 20 
2 adults 30 
3 or more adults 10 


Reporting standards not met. There are too few cases for a reliable estimate. 

1Attached structures include duplexes, townhouses, and rowhouses. Apartments include low-, medium-, and high-rise apart 
able to be determined for a very small number of cases, and these cases have been excluded from this analysis. 

2A few cases where mail was received in more than one way were categorized under "mail slot or mailbox attached to the hi 
received fit that category. Mail access type was not able to be determined for a very small number of cases, and these cases 
analysis. 

3These characteristics are based on self-reports provided by interview participants. 

NOTE: Percentages represent the percentage of interview participants that reported having opened at least one screener m; 
they were shown the NHES:2019 screener mailings. Participants that did not recall any of the mailings have been excluded 
that additional participants opened one or more mailings but either did not recall doing so or did not mention it during the 
different household member opened one or more of the mailings. In the small number of cases where more than one house 


Percentage that 
opened at least 
one mailing 


75.0 
80.7 


93.6 
67.7 


68.2 
96.8 
58.3 


ments. Structure type was not 


ome" since all the ways mail was 
have been excluded from this 


ailing during an activity where 
from this analysis. It is possible 
interview. It is also possible that 
hold member participated in the 


interview, this analysis focuses on the primary interview participant's reported behavior. Sample sizes are rounded to the nearest 5. Percentages are 


rounded to one decimal place but have not been changed to reflect sample size rounding. Details may not sum to totals due 


to rounding. 


SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Surveys Program (NHES), 2019. 
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Among the participants who remembered at least one of the mailings, 20 percent reported 
that they had not opened any of them. Just under half of these participants shared what they 
did with the unopened mailings. Most had kept at least one of the mailings with the intention 
of opening them later. 


Handling of opened mailings 


In contrast to participants who did not remember the NHES mailings or who did not open 
them, participants who opened at least one mailing saw the survey request and made a 
conscious choice about how to respond to it. Upon opening a mailing, participants took one 
of three actions: (1) they discarded the materials or actively decided not to respond, (2) they 
saved the materials for later or to give to someone else, or (3) they completed the survey. 
Similar to our analysis of remembering and opening mailings, we created an overall measure 
that summarized each participants’ handling of the opened mailings, and we classified 
participants into one of the three actions listed above.** Figure 5.2 shows the distribution of 
this behavior across all participants who opened at least one mailing. About three-quarters 
of them fell into the first two groups—that is, they did not end up completing the survey (this 
is equivalent to just under half of a// interview participants). 


Figure 5.2. Percentage distribution of interview participant handling of opened 
mailings among participants who opened at least one mailing: 2019 


Percent 
100 


80 
60 
40 


20 


Responded to the survey Saved the mailings Rejected or discarded the 
mailings 


Level of engagement with mailings 


NOTE: Details may not sum to totals due to rounding. Rounded number of eligible interview participants is 50. 
SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Surveys Program (NHES), 2019. 


24 Several participants had inconsistent reactions across the mailings. In such cases, we gave participants credit for their most positive 
reaction across all the mailings. Participants who lived in households that responded to the NHES were counted as having responded 
regardless of their other reactions to the mailings. Among all remaining participants, those who saved at least one mailing for later or to 
give to someone else were assigned to that classification even if they had also thrown away or rejected one of the other mailings. 
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Discarded the materials or actively decided not to respond. Fifty-six percent of the 
participants who opened at least one mailing indicated they had no intention of completing 
the survey and had thrown away the materials or had decided not to respond. These 
participants shared that they remembered receiving the materials sent in the screener 
packages and madea conscious decision not to participate in the survey. While some of them 
noted they had discarded the mailings quickly without reviewing them carefully, about half 
said they had taken a more deliberate and considered approach; the remainder did not 
discuss how thoroughly they had considered the request before deciding not to respond. In 
one exchange, a participant shared her reaction to receiving the second screener package: 


Participant: I did open this. Okay. 
Interviewer: You remember reading that? ... You remember that first line. 


Participant: Uh-huh [affirmative]... And I still didn’t do it because when I read 
it, | was like, ‘what survey education,’ and I was like, ‘Oh, this is nothing.’ I didn’t 
complete a survey anyway, and I tossed it. (7715) 


Like this participant, those who indicated that they had chosen not to respond to the survey 
shared a variety of reasons for making this decision, including their lack of interest or time, 
anti-government sentiments, and concerns over privacy. These reasons were distinct in the 
sense that they spoke to participants’ active intentions and decision-making in contrast to 
more passive factors that might have contributed to nonresponse, such as mail mis-delivery 
or participants forgetting to complete the survey. More details about participants’ reasons 
for choosing not to respond to the survey are included in section 5.2.3. 


Saved the materials to complete later or to give to someone else. Nineteen percent of 
participants who opened at least one mailing saved it to complete later or to give them to 
someone else to complete. These participants often sorted the mailings into a to-be- 
completed pile—or, as one participant put it, the “procrastination” pile (4373)—and 
demonstrated that they were open to completing the survey even though they never did so. 
One participant shared his reaction to receiving the fourth screener package in the FedEx 
mailer: 


If I’m not mistaken, I did just what I sat here and explained to you guys. I opened 
it up, looked at it, seen it was important, realized it was something that I needed 
to respond to, definitely wasn't interested in throwing it away or anything like 
that. I just set it aside, and I never got back to the Census stuff. I was actually 
thinking about what was I going to do? I know I’ve been getting that stuff for 
over a month now. I know it’s time to respond, and I knew that I was ready to 
respond. I just hadn't yet. (6393) 


These participants are distinct from those who chose not to complete the survey because 
they typically conveyed more interest in the mailings and signaled a willingness to complete 
the survey. As will be described in more detail in section 5.2.3, these participants mentioned 
several reasons for their reactions to the mailings. For example, some participants who were 
not native English speakers wanted to show the materials to someone else in the household 
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who had a better command of English (even if they had received bilingual materials) or give 
them to someone who had children (even if that person may not live in the household). 


Completed the survey. Finally, 25 percent of participants who opened at least one mailing 
lived in households that ended up completing and returning the survey after having been 
sampled for the qualitative nonresponse study but prior to having been interviewed.2° One 
participant said, “I filled it out within a day, but it took me a good week to mail it because I just 
kept forgetting it.” (7303) 


Subgroup variation in handling of open mailings. As shown in table 5.3, there was again 
some variation by subgroup in how participants handled the mailings after opening them. 
For example, participants who only had home internet access through a phone or tablet were 
more likely to choose not to complete the survey than were participants who had home 
internet access through a computer. Spanish speakers were more likely than English 
speakers to save the mailings for later. Participants with higher household incomes were 
more likely than those with lower household incomes to end up responding to the survey. 
For additional subgroup findings, see appendix D. 


25 Two participants whose household had returned the survey (according to their address’s final NHES:2019 outcome code) did not 
remember any of the NHES mailings during the materials review activity. Although the reason for this discrepancy is unclear from the 
data, it could be that the two participants forgot they returned the survey. It also could be that someone else in the participants’ 
respective households completed and returned the survey without the participants’ knowledge. Because the focus of this analysis is on 
participants’ reactions to the mailings during the interview, these two participants are not included here. We instead elected to code 
these two cases as “not remembering” in this analysis. In contrast, one participant who conducted the short interview shared that (s)he 
had completed the survey, but a completed screener was never received for this address. 
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Table 5.3. Percentage distribution of interview participant handling of opened 
NHES:2019 screener mailings among participants that opened at least 
one screener mailing, by selected characteristics: 2019 


Number of 
interview 
participants that 
opened at least 


Percentage 


one screener that responded 


to the survey 


Percentage 


Percentage that rejected 
that saved or discarded 


the mailings 


the mailings 


Selected characteristics mailing 
Total 50 
Nonresponse study site 
Texas 10 
Connecticut 15 
California 10 
Ohio 15 
Nonresponse study interview language 
English only 40 
Spanish only 5 
Mix of English and Spanish + 
Observed structure type! 
Single-unit 30 
Attached 10 
Apartment 15 
Observed mail access type? 
Mail slot or mailbox attached to the home 25 
Mailbox at the end of the driveway, across 
the street, or at the end of the road 15 
Mailbox, slot, or room in multi-unit building $ 
No mailbox or slot in view 5 
Age? 
18-24 a 
25-34 10 
35-44 15 
45-54 10 
55-64 10 
65 and older 5 
Refused 
Gender? 
Male 20 
Female 30 
Refused $ 
Race/ethnicity? 
White, non-Hispanic 15 
Black, non-Hispanic 15 
Hispanic 15 
Other race, non-Hispanic + 
Refused $ 


25.0 


16.7 
28.6 
20.0 
31.3 


23.8 


19.2 


25.0 
21.4 
20.0 
12.5 


16.7 
42.9 


55.8 


58.3 
50.0 
60.0 
56.3 


59.5 
42.9 


See notes at end of table. 


101 


Chapter 5. Experiences with and Opinions About Surveys 


Table 5.3. Percentage distribution of interview participant handling of opened 
NHES:2019 screener mailings among participants that opened at least 
one screener mailing, by selected characteristics: 2019—Continued 


Number of 
interview 
participants that Percentage 
opened at least Percentage Percentage that rejected 
one screener that responded that saved or discarded 
Selected characteristics mailing tothesurvey themailings the mailings 
Education? 
High school or less 25 12.5 20.8 66.7 
Some college, but no bachelor's degree 15 28.6 21.4 50.0 
Bachelor's degree 10 37.5 + 50.0 
Graduate degree 5 50.0 + 33.3 
Enrollment status? 
Enrolled 5 33.3 # 66.7 
Not enrolled 45 23.9 21.7 54.4 
Refused # # # # 
Employment status? 
Employed for pay 35 27.8 13.9 58.3 
Not employed for pay 15 18.8 31.3 50.0 
Household income? 
$30,000 or less 10 £ 41.7 50.0 
$30,001-$60,000 15 25.0 # 75.0 
$60,001-$100,000 10 37.5 25.0 37.5 
$100,001 or higher 10 50.0 $ 40.0 
Refused 5 # 33.3 66.7 
Language spoken most often by adults in 
household? 
English 40 23.1 18.0 59.0 
Spanish 10 22.2 33.3 44.4 
English/Spanish equally + + + + 
Other $ + $ $ 
Home internet access? 
No access + + + $ 
Phone/tablet access only 10 # 27.3 72.7 
Computer 40 33.3 18.0 48.7 
Refused 
Child in household? 
Yes 25 21.7 17.4 60.9 
No 30 27.6 20.7 51.7 
Number of adults in household? 
1 adult 15 13.3 40.0 46.7 
2 adults 30 33.3 10.0 56.7 
3 or more adults 5 £ £ 71.4 


#Rounds to zero. 

+Reporting standards not met. There are too few cases for a reliable estimate. 

1Attached structures include duplexes, townhouses, and rowhouses. Apartments include low-, medium-, and high-rise apartments. Structure type was not 
able to be determined for a very small number of cases, and these cases have been excluded from this analysis. 
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2A few cases where mail was received in more than one way were categorized under "mail slot or mailbox attached to the home" since all the ways mail was 
received fit that category. Mail access type was not able to be determined for a very small number of cases, and these cases have been excluded from this 
analysis. 

3These characteristics are based on self reports provided by interview participants. 

NOTE: Participants who "responded to the survey" are those whose household ended up responding to NHES:2019. Participants who "saved the mailings" 
are those that reported saving at least one mailing for later or to give to someone else. Participants who "rejected or discarded the mailings" are those that 
reported rejecting the request to participate or discarding at least one mailing. Participants who saved some mailings and rejected others were placed in the 
"saved the mailings" group. In the small number of cases where more than one household member participated in the interview, this analysis focuses on the 
primary interview participant's reported behavior. Sample sizes are rounded to the nearest 5. Percentages are rounded to one decimal place but have not 
been changed to reflect sample size rounding. Details may not sum to totals due to rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Surveys Program (NHES), 2019. 


5.2.3 Reactions to the NHES Materials 


During the materials review activity, participants discussed in detail why they remembered 
the mailings and why they engaged with them in the way that they did. They described 
general features of the screener packages that were true across all five mailings, as well as 
mailing-specific factors that influenced their decision-making at each point along the 
recall/engagement process discussed in section 5.2.2. Participants also described some 
general factors that were unrelated to the design of the screener packages (e.g., being too 
busy). This section summarizes key themes in participants’ comments, both overall and for 
specific subgroups of interest. 


General NHES design features 


In describing what influenced their recall of and engagement with the NHES mailings, 
participants pointed to several general features of the NHES design that held true across all 
of the mailings: the officialness and government affiliation, the Census affiliation, the 
“Member of [city] household” mailing address, the education survey topic, the lack of 
requirement to complete, the personal nature of the survey questions, the internet response 
mode, the use of bilingual materials, and the perceived survey length. These features are 
described in greater detail below, in order from most to least commonly mentioned. 


Officialness and government affiliation. Most participants discussed the official nature of 
the mailings (i.e., “officialness”). Participants specified that several features of the envelopes 
conveyed officialness, including the bold font, the use of pre-paid postage, and the “Official 
Business” and the “Penalty for Private Use” text on the non-FedEx mailings. In addition, 
participants shared that they thought the barcode conveyed officialness. As one participant 
said, “I just know that when I see a barcode, I know it's important. I’ve never seen a barcode on 
mail that really wasn't important...Whether I have time for it or not, whether I'm interested or 
not, whether it has my name on it or not, I respect the barcode.” (6393) 


When discussing whether they remembered receiving the mailings, some participants 
shared that the official nature of the mailings sparked their memories because they had 
mentally marked them as being important. Similarly, the most common reason participants 
gave for why they decided to open at least one mailing was because the mailings looked 
official and thus important enough to open. One participant shared, “the officialness of it, I 
always look at official mail. I mean, I’m going to sit down there and really have the time to really 
look at it.” (6535) Other participants remarked that the mailings looked like bills or a court 
summons, which made them more likely to open them. 
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For participants who did not have time to address the survey request when they opened the 
mailings, the official nature of the mailings was a factor in deciding to save them to address 
at a later time. One participant shared, “So, right there. I had that little crate, and that’s on my 
mail that I feel like is important that I don’t shred and don’t rip up. So, it’s in here.” (4399) 


In addition to the general feeling that the mailings were official, participants also perceived 
that they were affiliated with the federal government. Participants most often inferred that 
the mailings were affiliated with the government because of the return address (i.e., the U.S. 
Department of Commerce) and the Census Bureau logo on the front of the envelopes. For 
participants who were more inclined to open official-looking mail, the mailings’ affiliation 
with the government motivated them to open the mailings, as the following participant 
shared: 


Participant: It does look like it’s a more official business, you know, the U.S. 
Department of Commerce. 


Interviewer: What do you think when you see something that says, the U.S. 
Department of Commerce?” 


Participant: I would say I would open it... It’s about kind of the whole premise 
of how it’s on the envelope...the bold print, right? Stands out to me, right? Official 
business, penalty for three hundred dollars... I think it says, “You should probably 
open me.” Right? (7329) 


Depending on their overall attitudes toward the government, the mailings’ association with 
the government motivated some participants to engage with the mailings but discouraged 
others from doing so. As described in chapter 3, among the half of participants who explicitly 
shared their thoughts on the federal government, most expressed negative views. The 
minority of participants who had positive views of the government tended to perceive the 
mailings as official and legitimate and engaged with them accordingly (i.e., tended to open 
and read the mailings). While participants across all racial/ethnic groups mentioned the 
mailings’ government affiliation as their main motivation to open them, Black participants 
tended to mention this factor most consistently. Likewise, a few participants who eventually 
completed and returned the survey shared that the mailings’ affiliation with the government 
motivated them to do so. As one participant said, “In general, anything at the federal level or 
something that [I] deem as important, we need to fill out.” (7303) 


On the other hand, some participants who expressed anti-government sentiments shared 
that they declined to respond to the survey because of the mailings’ association with the 
government. Around half of these participants had a general openness toward surveys but 
were not willing to participate in government-sponsored surveys. The other half felt that 
responding to government-sponsored surveys was unnecessary because they believed that 
the federal government already had access to their information. 


Hispanic and Spanish-speaking participants more often noted that they had declined to 
respond to the survey because of their anti-government views. Many of these participants 
shared that they were scared to receive mail from the government and were generally 
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reluctant to give out their information on any kind of government-sponsored survey. As 
described in chapter 3 and in the first half of this chapter, for example, several participants 
expressed fears that filling out the NHES could lead to negative consequences related to 
immigration and their legal status. A few Hispanic participants specifically mentioned that 
they perceived the government as being hostile toward the Hispanic and/or Mexican 
community and were thus apprehensive about receiving and responding to government- 
affiliated mail like the NHES mailings. One participant shared: 


Recently it came out on television; they were saying that the government was 
sending [something out]... but really, we shouldn't fill it out because it seems to 
be a matter of immigration to have people located...a kind of census, but by 
immigration. So, they said not to answer anything until they have investigated 
what the purpose [of] it is. (4711) 


These findings are consistent with those in chapter 4, as well as in section 5.1.3, suggesting 
that participants’ decisions are shaped by the survey sponsor and mailing sender. 


Census affiliation. Similar to the effect observed for “officialness,” many participants noted 
that the mailings’ association with the Census (as conveyed by the Census Bureau logo on 
the front of the envelopes) sparked their memories because they had mentally marked the 
mailings as being important. For example, one participant said, “J remember getting one of 
those. And I remember...the Census. I done heard of it before... !remember getting a letter from 
them before.” (7057) 


As discussed in chapter 3, participants shared a range of levels of familiarity with and 
attitudes toward the Census; these perceptions tended to shape the ways in which they 
reacted to the mailings’ Census affiliation. Those who had generally positive views about the 
Census and conveyed an understanding of its importance tended to share that the NHES 
mailings’ association with the Census motivated them to open the mailings. One participant 
shared that he had opened the mailing “because it’s from the Census Bureau—And if [the 
Census] was over with, I would not open it.” (6175) As this participant implied, several 
participants suggested they had opened the mailings because the Census Bureau logo on the 
envelopes made them believe they were being asked to fill out the Census. 


As noted in chapter 3, late NHES respondents expressed a higher degree of enthusiasm about 
the Census than did final nonrespondents. For these participants, the NHES mailings’ 
association with the Census Bureau was a motivating factor for them to engage with the 
mailings, as the following participant explained: “I do believe in filling out local censuses and 
federal censuses... I see a U.S. Census bill here, so it looks legit. I would open it and see what it 
is.” (5195) 


On the other hand, participants who expressed suspicion of or harbored negative attitudes 
toward the Census indicated that the mailings’ perceived association with the Census 
prompted them to ignore the mailings or discard them without opening them. As noted in 
chapter 3, participants who did not have a bachelor’s degree were more likely to perceive 
the Census negatively compared to participants whose highest level of education was a 
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bachelor’s degree or more. Most of these participants shared that they had privacy concerns 
around sharing their personal information. 


In some cases, the mailings’ Census affiliation contributed to participants misunderstanding 
what the survey was about. One participant shared, “J did read it... and then I realized that I 
probably had the wrong idea of what it was about... Because initially like I told you earlier, I 
thought that when it said Census, I thought it had more to do with our household... But it’s more 
focused on education.” (4399) Another participant was initially concerned that the mailings 
were a scam because they came from the Census Bureau in a non-Census year (4025). She 
said, “The Census [is] done every 10 years and it’s not a time for that to be happening... That’s 
what threw me about letters like this ... I've thrown [out], I can’t tell you how many letters from 
you folks that I hadn’t open[ed].” 


A few participants who had opened the mailings expecting the content to be related to the 
Census shared that they felt tricked. One participant shared, “See, that’s what I’m saying. 
That’s why I won't open too much mail because... it’s not even actually from the Census Bureau. 
You see? ... They’re just using their titles and everything to draw attention for it to get opened. 
So, once I take it out, I would know. As soon as I see the education survey, I’d be done with it 
because education surveys don’t ever end either.” (6175) 


“Member of [City] household” mailing address. Consistent with the chapter 4 findings 
from the mail sorting activity, the most common reason why participants said they chose not 
to open the NHES mailings was because the mailings were not directly addressed to them. 
These participants explained that they assumed that the mailings, which were addressed to 
“Member of [City] household,” were not important enough to open. This sentiment tended 
to be shared by participants who lived in households without children or were Spanish- 
speaking. 


Other participants shared that, since the mailings were not directly addressed to them, the 
mailings felt generic and impersonal. One participant shared, “So this one [pressure sealed 
envelope] I would have tossed automatically. It just seems like nothing to me... This seems more 
like something that was mass-produced than something personal.” (7715) 


While these participants followed a general rule of not opening mail that was not directly 
addressed to them, some made exceptions for mail that looked important. In some cases, the 
officialness of the return address convinced participants to open the mailings despite the 
lack of a personalized mailing address, “Okay, so I’m looking at this [initial screener package], 
and I would say this looks like something important. But, like I said, it’s addressed to ‘Member 
of Household.’ But I would be curious enough to open it up because you have ‘U.S. Department’ 
on there.” (7715) This perspective also helps explain why the majority of participants noted 
during the example mail activity that they would engage with the initial screener package 
even though it was not addressed to them personally. 


Multiple mailings. Several participants shared that they remembered the mailings because 
they had received more than one. One participant said, “As a matter of fact, now that I think 
about it, |do remember getting a second one of these, at which time I asked my grandson [about 
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it]. lshowed him [the pressure-sealed envelope], and I said to check it out to make sure this was 
really who it says it is.” (4167) 


A few participants shared that receiving multiple mailings from the same return address 
conveyed to them that the mailings were important to open. As the following participant 
explained: “I just kept getting [the mailings], so I just opened one of them. So yeah, I remember 
getting one of those... It just kept coming, so I said, let me open it.” (7057) Some of the 
participants who completed the survey shared that receiving multiple mailings influenced 
their decision to do so. For them, receiving follow-up mailings reminded them to complete 
and return the survey, which was their original intention. One participant recounted: 
“Everything is about timing here. I may have gotten a second [mailing]. And then I got the [the 
third screener package], and I’m like, ‘Oh! They’re really hounding me.’ Forget it. Everything 
else stops. I’m calling them right now.” (7015) 


However, a few participants, almost all of whom were male, shared that they were less 
inclined to open subsequent mailings after the initial screener package because they felt like 
they had already decided whether to participate in the survey. One participant said, “J 
wouldn't even [have] opened this one [the pressure-sealed envelope] ... because I probably 
would think in my head it’s the same thing as the one that I opened before.” (4153) 


Education survey topic. Consistent with the findings in section 5.1.3 that the salience of the 
survey topic affects participants’ willingness to engage with surveys, several participants 
reacted to the survey materials being related to education. Both the education references in 
the letters and the photos on the cover of the paper screener were referenced as indicators 
that the survey was about education. 


Many of these participants shared that they remembered the mailings because of their focus 
on education. For example, when the following participant reviewed the initial screener 
package during the interview, the mention of education in the cover letter sparked his 
memory of having received it before: “Want me to read it?...‘Important national survey 
education.’ Okay. This is like something that [I] got.” (7401) 


Additionally, many participants who completed the survey explained that they did so 
because the mailings were related to education. These participants generally expressed two 
types of support for or interest in education. First, education was a salient topic for some 
because of their own personal experiences in the education system or because they had 
school-age children or had a relationship with school-age children (e.g., nieces/nephews, 
grandchildren, neighbors). These participants tended to frame their motivation to complete 
the surveys around their children’s education and/or the local school system. Second, some 
participants demonstrated a more general support for education-related issues in explaining 
what motivated them to complete the survey. 


Among the participants whose household completed the screener as late respondents, White 
participants tended to more consistently stress that their main motivation for completing 
and returning the survey was because they supported education, both personally and in 
general. One shared, “It’s like I said, my three kids went completely through public school. My 
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[other] son just went to private... I [fill] this out...and [that] would maybe help. I mean, it’s 
always good when you can help schools and try to figure it out.” (7583) 


In contrast, a few participants indicated that they had little interest in education and that, as 
a result, the survey was not relevant to them. Of these participants, those who did not have 
school-age children tended to assume that because the survey was related to education, it 
was not relevant to them. One participant explained, “I didn’t even read it. I mean, I think I 
saw what it was and put it aside. Actually, since I’m not involved in child care, it didn’t appeal 
to me one way or the other.” (7149) 


Two participants indicated that they had given the survey materials to someone else for 
whom they thought it would be more relevant. One participant shared that, because she 
assumed that it was not relevant to her, she passed one of the mailings on to a neighbor in 
her building: “I sent this letter upstairs. Because when I saw that it said small children, she has 
lots of small children and I thought it was for her... it has to be for them because they have kids 
in school, [so] I thought it was for them.” (5429) These responses indicate that not all people 
sampled for NHES noticed the instructions that asked sample members to complete and 
return the survey even if there are no children in the household. Notably, participants’ 
reactions to the survey’s education topic (i.e., interested in education, felt that education was 
irrelevant) did not seem to vary based on their level of educational attainment. 


Lack of requirement to complete. A few of the participants who had opened at least one of 
the mailings indicated that they did not read the cover letters carefully, but rather scanned 
them to search for “the ask,” i.e, what they were required to do (e.g., pay a bill). In the 
following exchange, one participant described how he processed the initial screener package 
when he received it: 


Participant: I really don’t read it like that, I just scan. 
Interviewer: What are you scanning for? 


Participant: For if I owe something or if I have to do something very important. 
(6081) 


Typically, when these participants discovered that they were not required to do anything, 
they discarded the mailings. Participants who did not have a bachelor’s degree were 
particularly likely to say that they would only be inclined to respond to a mailing if it was 
something that required action, like a bill or a jury summons. Moreover, a few participants 
shared that, because the early mailings lacked any specific deadlines for completing the 
survey, they felt little urgency to do so. As the following participant explained: “/] put the 
mailing away] waiting...to have the time to do it. Because...I didn’t see a deadline or anything 
on it. But that’s why I said I didn’t bother to read...it.” (4399) 


Personal survey questions. Consistent with the general privacy concerns discussed in 
section 5.1, some participants who were parents of school-age children said they had decided 
not to participate in the NHES because they were unwilling to share any information about 
their children. One participant who had children explained, “They’re asking for too much. If 
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it’s a male or female, they're homeschooled, college, not in school. Why do they want to know? 
No. No, I wouldn’t even answer it... | don’t want them having my kids’ information.” (6331) 


Most of these comments were shared as a reaction to the paper screener included in the third 
and fourth screener packages because, unlike with the survey URL shown in the earlier 
mailings, participants could see and assess on the paper screener the type of information the 
survey was requesting of them. For participants who had acute concerns about sharing their 
children’s information, this privacy concern seemed to override other considerations that 
may have otherwise inclined them to participate, such as their interest in or support for 
education or their general belief in the importance of surveys. 


Internet response mode. A few participants indicated that they could not or preferred not 
to complete the survey online because they had limited to no access to the Internet, were 
uncomfortable using computers/browsing the Internet, or did not want to share personal 
information online. The responses followed the same patterns about Internet concerns 
discussed earlier in section 5.1.3. 


One issue about completing surveys on the Internet came up only in context of the NHES 
materials. A few participants who only had access to the Internet on their cell phone 
expressed reluctance to do the survey online. One participant explained that she tried to do 
so but was unable to access the survey through her phone (5429). Those who called the toll- 
free phone number rather than access the survey via the web link found this option to be a 
helpful workaround. As the following participant said: “I think when I received this...I called 
the phone number rather than going online. I’m a little bit non-computer, tech-savvy stuff.” 
(7015) 


Bilingual materials. Some Hispanic and Spanish-speaking participants shared that being 
sent clearly worded, easily understood Spanish-language materials motivated them to 
complete and return the survey. For example, one Hispanic participant said, “/ sent this one 
because it came in Spanish, and I could read it well... maybe these [other mailings] didn’t come 
to me in Spanish.” (5305) Another participant shared that she appreciated that the Spanish 
translation of the mailing content was a high-quality translation compared to other bilingual 
mailings she typically receives (5195). 


As noted above, Hispanic and Spanish-speaking participants saved the mailings more often 
than did other participants. Some of these participants shared that they kept the mailings to 
pass on to other members of their household who had a better command of English, even 
when the mailings also included Spanish materials. The most common example of this was 
Spanish-speaking parents who saved the mailings for their English-speaking children as the 
following participant recounted: “This one [third screener package], it caught my attention. I 
have it there, I didn’t throw it away. [I took] a photo [and sent it] to my daughter.” (4187) 


Perceived survey length. Participants’ willingness to complete and return the survey 
depended on their perception of how long the survey would take to complete. For example, 
a few participants did not respond because they did not see the information in the materials 
about how long the survey would take to complete. Without this information, these 
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participants assumed that the survey would take too long. One participant stated, “/But] it 
makes me laugh [now] because I put it aside to fill it out because I thought it was going to be 
this long thing.” (4399) 


Some of the participants who completed and returned the survey indicated that they did so 
because they did not expect the survey to take a lot of time. These participants said they had 
learned about the survey’s anticipated length in one of three ways: (1) by visiting the web 
instrument; (2) by looking through the paper screener; or (3) by reading the cover letter 
included in the later screener packages, which stated how long the screener takes to 
complete. Brevity was particularly relevant for some participants who did not have children 
as this exchange shows: 


Interviewer: You did send [the survey] back? 


Participant: Yeah. All I had to do was check [one box] because I don’t have any 
kids. If I had kids, I had to go check other boxes. (5441) 


Mailing-specific influences on engagement with and reaction to NHES mailings 


In addition to the general factors above, several mailing-specific features influenced 
participants’ recall of or engagement with the mailings. These factors are summarized below, 
in order from most to least salient to participants. 


Five-dollar incentive (initial screener package). Across all the mailings, the incentive that 
was included with the initial screener package was among the most frequently referenced 
elements that participants said they remembered. Most participants who remembered the 
initial screener package shared that the surprise and novelty of receiving $5 jogged their 
memory of having received it. 


For most of the participants who mentioned it, the incentive was a positive factor in their 
reactions to the mailings. Several late respondents shared that the incentive made them feel 
obligated to respond. One shared, “/The incentive] guilted me ... I felt very guilted. Like, Oh my 
God! I can’t just take their money and not call them and see if there’s something [I should do].” 
(7015) White and Black participants were more likely than participants from other 
racial/ethnic groups to share that the five-dollar incentive was a motivating factor for them 
to engage with the mailings. The participants who had a positive reaction to the incentive 
but did not respond to the survey did not offer an explanation as to why the incentive was 
not sufficient to motivate them to respond. 


A few participants reacted negatively to the cash incentive, which discouraged them from 
completing the survey. These tended to be Hispanic participants, participants without 
children in the household, and participants who did not have a bachelor’s degree; they 
expressed negative or suspicious attitudes about receiving cash through the mail. Many of 
these participants suspected that the mailings could bea scam to steal their information. One 
participant explained, “] think that as soon as I saw it: ‘they want something from me to have 
sent five dollars’... Nobody gives anything for free. There’s always a reason...Or those scams like: 
‘Now, I send you this, but you pay the one-time fee for I don’t know what.” (4711) In another 
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exchange, a Vietnamese participant described the five-dollar incentive as “sketchy money,” 
explaining that randomly receiving cash was seen as a kind of curse in her culture. 


Envelope design/appearance. The appearance of certain envelopes seemed to stick in 
participants’ memories, as well as influencing their decision on whether to open or discard 
the mailings: the pressure-sealed envelope, the FedEx mailer, and the larger envelopes used 
for the third and fourth mailings. 


Pressure-sealed_ envelope. Many participants who remembered the pressure-sealed 
envelope indicated that they did so because the envelope’s unique design made it stand out 
from more conventional mail. 


Among those who remembered opening the pressure-sealed envelope, most indicated that 
the envelope’s unique design made them think the contents were important, making it more 
likely for them to open it. Participants who lived in households with children tended to 
express this view at more often than did those who lived in households without children. 
Likewise, participants who did not have a bachelor’s degree tended to perceive pressure- 
sealed envelopes as being important more often than did participants who had a bachelor’s 
degree. These participants tended to share that the pressure-sealed envelope reminded 
them of other types of important or official mail, such as W2s, checks, tax documents, or court 
summonses. This then encouraged participants to open the mailing because they perceived 
these types of mailings as important. 


However, a few participants explained that they threw away the pressure-sealed envelope 
without opening it because they have had past experiences with this type of mailing turning 
out to be a scam. Some of these participants also shared that they disliked the design of the 
pressure-sealed envelope. As one participant said, “[The pressure-sealed envelope] so easily 
gets lost within all of this... I’m just going to be honest, unless I know it’s a check, it’s just kind 
of a pain to open, so I probably wouldn't.” (4209) 


FedEx mailer, Similar to the way participants reacted to the unique design of the pressure- 
sealed envelope, most participants commented that they associated FedEx packages with 
important or official mail. In particular, Black participants, participants who lived alone, and 
male participants were more likely to note that they would open the FedEx mailing because 
they thought it contained important content. In addition, one late respondent described how 
the FedEx mailer sent as part of the fourth screener package helped convince her to not only 
open the mailing but to complete the survey: 


I receive a lot of stuff from FedEx. But, once again, if you see FedEx it means 
something serious, you know. Nobody’s going to take the time to send you a 
FedEx and all that just because of nothing...And right here we got the same thing. 
Reminder that I have to do a survey that’s not going to take more than three 
minutes and I have to do it. Period. (4271) 


A few participants indicated that they threw away the FedEx mailings because they were not 
expecting to get anything via FedEx, as the following participant said: “When FedEx comes to 
my mind, that’s like something you've ordered. If I know I’m not expecting [it], I wouldn’t open 
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it.” (4495) Like participants’ attitudes toward the pressure-sealed mailing, these participants 
expressed concern that the FedEx mailing was sent to them in error or as part of an elaborate 
scam. 


Larger envelopes. Participants also tended to remember the large size of the USPS First Class 
envelopes used for the third and fourth screener packages. The large envelope size made 
them think the mailings were important and helped them stand out from other, more 
conventional mail. Some participants also noted that these envelopes reminded them of 
other important mailings they have received in the past, such as applications and Social 
Security or Medicare/Medicaid documents. Others shared that the large size of the envelopes 
motivated them to open the mailings. As one participant described: “The bigger envelope? I 
would read it because it looks like it would be more important. I probably have something to 
read.” (5449) 


Finally, a few participants shared with the interviewer that the “please respond within two 
weeks” text on the envelope for the fourth screener package motivated them to open and 
respond to the mailing. For example, one participant said, “I saw like, ‘Please respond within 
two weeks.’ So that gives it like a little bit sense of urgency that this could be something that I 
need to pay attention to.” (5685) Some of these participants found this text to be helpful 
because it narrowed down a concrete timeframe for them to complete the survey. 


Cover letter contents. Some of the participants shared their reactions to aspects of the 
cover letters that varied across the mailings. Overall, most participants thought that the text 
on the cover letters was clear and direct. One participant said, “/The cover letter] is pretty 
self-explanatory, and it’s very clear. Cut straight to the chase. So, I’m used to things like that.” 
(5265) Participants also highlighted certain aspects of the cover letters used for the second, 
third, and fourth screener packages that they either liked or did not like. 


Second screener package. For the second screener package cover letter, a few participants 
discussed the bulleted list of education statistics included at the top. In one exchange, a 
participant who completed the survey remarked how he liked that these statistics were 
included, noting: “I feel like facts are always helpful in marketing... For me, analytical mind, I 
like facts in general. (7303) \n contrast, a few participants who had shared that they were not 
interested in education or did not have school-age children living in the household indicated 
that the inclusion of these education statistics made them feel as if the survey was not 
relevant to them and thus made them less interested in completing it. A few of these 
participants remarked that the statistics made the letter feel cluttered with too much 
information before getting to the main point—the survey request. 


Third screener package. For the third screener package cover letter, some participants 
mentioned that they were struck by the text that read “Did you know? All households that 
were Selected for the survey that do not respond harm the accuracy of the study’s findings.” 
One participant shared how this text gave her a sense of urgency to complete and return the 
survey, sharing: “That would probably [motivate me] to get this done... Just because I feel 
somewhat responsible... I don’t want to harm the accuracy of what they’re trying to do.” (6587) 
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Other participants had a negative reaction to parts of the third screener package cover letter. 
These participants shared that they would have been more willing to complete the survey 
had they not been immediately turned off by the perceived negativity of the text. For 
example, a few participants were turned off by the tone conveyed in the first sentence of the 
cover letter (“We have not yet received your completed survey for the 2019 National 
Household Education Survey”). In the following exchange, one participant commented on 
why she reacted negatively to this text: 


Participant: It’s a reprimand. You haven't taken care of this yet. 
Interviewer: Does that make you more or less likely to want to do it? 
Participant: Probably less. 

Interviewer: Okay. Does it feel like you’re being chastised or punished? 


Participant: Mm-hmm [affirmative]. Just chastised...with a slap on the wrist. 
You didn’t do this yet...Then I feel guilty. And I don’t like feeling guilty, so then I 
just don’t want to deal with it. (7355) 


Fourth screener package. For the fourth screener package, a few participants who had 
completed and returned the survey commented that the text that read “It may take 3 minutes 
or less to complete the survey” helped convince them to respond. In line with the finding 
above that being able to tell that the survey was low burden helped convince participants 
who did not have children in their households to complete the survey, these participants 
shared that this text helped set their time expectations in a way that made them more likely 
to take the survey. 


Finally, and consistent with the finding above that a few participants reacted positively to 
the “Please respond within two weeks text” on the envelope of the third and fourth screener 
packages, a few participants commented that this same text, which was included on the cover 
letter of the fourth screener package, was a helpful inclusion. This text conveyed a sense of 
urgency and provided participants with a concrete timeframe to complete and return the 
survey. 


Commonly Asked Questions (CAQ) Enclosure (all mailings other than pressure-sealed 
envelope). Some participants said that they either ignored or skimmed the CAQ enclosure 
because it contained too much text that they were not inclined to take the time to read, as 
the following exchange highlights: 


Participant: Good question[s], good answer |s]. Again, a little extensive if you 
just want quick feedback. 


Interviewer: Too much writing. You’re not going to read all that right there. 


Participant: Nobody’s going to. (7465) 
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In contrast, another participant wondered aloud if the document contained enough 
information, “To a certain degree [the CAQ enclosure] probably would [help] but not everyone 
has the same questions. I learned that the hard way... Everybody doesn’t have the same 
question. And everybody’s not going to be willing to go the extra mile to ask that.” (7075) The 
few participants who did review the document indicated that they skimmed it rather than 
read it in full. None of the participants, however, commented that the document provided 
specific information that made it more or less likely for them to complete the survey. 


Paper screener (third and fourth screener packages). The paper questionnaire included 
as part of the third and fourth screener packages garnered a range of reactions from 
participants. Some participants remarked on the photographs included on the cover of the 
questionnaire. These pictures appeared to convey to most of these participants that the 
survey was focused on education. Participants who had school-age children or expressed 
belief in the importance of education tended to respond favorably to these pictures. 
However, in line with other information about households without children, some 
participants who did not have school-age children automatically interpreted the pictures to 
mean that the survey was not relevant to them. In these instances, participants said the cover 
letter image led them to discard the mailings without carefully reading the other materials 
that were included in the mailings. 


The questionnaire format also garnered mixed reactions. Some had positive reactions, as did 
the following participant: “J think [the paper questionnaire] looks pretty simple and easy 
actually... I would fill this out.” (7583) However, other participants criticized the form for 
looking too complex. For example, one participant said, “J mean, seriously... [the paper 
screener] looks like a tax return, almost like I'll start here and then we go up to there and then 
come over here and go down that way. I mean, we’re flowing all over the place.” (7329) These 
participants seemed to initially skip over the instructions and cast their immediate attention 
to the fields requesting the names of the children living in the household. Several participants 
who did not have children then concluded that this survey was not relevant to them and 
discarded it. 


Some participants liked being able to see the questions laid out on the paper screener. One 
shared that the paper format enabled her to quickly gauge how long the survey would take 
to complete (5305). Another explained, “The paper one, I could read the questions, and I knew 
what they were, so that’s what made me fill it out. When I looked at the online thing, I didn’t 
really know what it was, so I didn’t get to it right away.” (7303) As the exchange continued, 
this participant elaborated that being able to see the questions helped her determine what 
information she was comfortable giving out: “I liked that it gave the option [to provide initials] 
because I didn’t want to put my kids’ names on it, so I actually did initials. I felt better about 
that, not giving personal information.” 


In contrast, and as described above in section 5.1.2, some participants were uncomfortable 
about providing information about their children, particularly their names, as the survey 
requested. These participants did not seem to be aware that the paper screener gave them 
the option to provide initials or a nickname to protect their identities. Regardless, 
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participants who demonstrated an acute concern about their children’s personal 
information tended to discard the paper screener without completing it. 


General factors unrelated to NHES design 


Finally, participants described a number of reasons unrelated to the NHES design that 
influenced their recall of or engagement with the mailings. These reasons can be summarized 
under two general factors: everyday life/busyness and general habits. 


Everyday life/busyness. A few participants shared that the extent to which they engaged 
with the mailings depended in part on their daily habits, struggles, or the overall busyness 
of their lives. Comments related to the NHES mailings and busyness mirrored those provided 
in chapter 3. For example, some participants shared that they had saved the mailings to 
address later but simply forgot because they were busy. One participant said, “J was going to 
[do this], but... I literally forgot. I just put it up there, then guess I forgot that I was supposed to 
read this.” (7275) Others noted that they chose not to complete the survey because they felt 
they did not have time to do so. 


General habits. Some participants’ reactions to the mailings appeared to be driven by 
general habits or the feeling that “this is what I always do.” Some participants shared that 
they opened the mailings simply because they were in the habit of opening most mail they 
receive. One late respondent shared that he chose to complete the survey because “I'll fill out 
just about anything.” (6287) In contrast, some participants indicated that they did not 
complete the NHES because, as a rule, they do not do any surveys. As one said, “I don’t fill out 
surveys. What’s the word? Apathy? Just, I see a survey, I don’t know if it’s because of laziness or 
lack of interest [or], like I said, apathy.” (4655) These participants tended to be more set in 
their mail processing patterns and attitudes toward surveys and, in turn, seemed more 
unlikely to be swayed one way or another by general or survey-specific design features. 
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Chapter 6. Nonrespondent Typologies 


The purpose of this chapter is to synthesize the findings presented in chapters 3 through 5 
in a way that helps better understand the different factors that led interview participants not 
to respond to the NHES prior to the fourth screener package. Based on participants’ reported 
experiences and attitudes and on address observations conducted prior to the interview, we 
created seven nonrespondent typology groups that best explain the barriers to response. 
While the themes presented in earlier chapters cover a wide range of experiences, behaviors, 
and attitudes, the typologies identify the factors that were the most salient for individual 
participants’ NHES response decision. Segmenting participants into these typology groups 
can offer insight into what themes to highlight—or avoid—when trying to improve survey 
response rates. We also explored whether there were patterns in the characteristics of the 
individuals in each group. In cases where participants in a typology share similar 
demographic characteristics, it may be possible to target data collection protocols or 
messages specifically to those subgroups for which they will be the most effective. 


A similar exercise was conducted as part of a 2019 U.S. Census Bureau study in preparation 
for the 2020 Census. Their six psychographic profiles (“mindsets”) were used to help guide 
outreach messaging about the 2020 Census (see Kulzick et al. 2019 for more information). 
Although the Census study was more quantitative in nature than the NHES qualitative 
nonresponse study, it provides support for the general concept of creating nonrespondent 
typologies. Using the Census mindsets to explain the factors that drive NHES nonresponse 
was not feasible since the scope and purpose of the Census and the NHES are vastly different. 
To that end, this chapter outlines the attitudinal, behavioral, and demographic 
characteristics that comprise each NHES nonresponse typology. 


6.1 Creation of Typology Groups 


Based on the themes that emerged in chapters 3 through 5, we identified a set of key factors 
on which interview participants varied. For each factor, we then assigned each participant 
to a category based on the information that was shared during the interview or noted during 
the address observation. For example, for the “mail checking routine” factor, we assigned 
participants who said they check their mail every day to a “check mail frequently” category. 
Exhibit 6.1 lists the factors that were identified and the categories that were included in each 
factor. 
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Exhibit 6.1. Factors used to create typology groups, by topic area 


Topic area | Factors (categories) 


Time use 


Type of paid labor (arts, education, executive, education, manual labor, service/retail 
industry) 

Busyness (extremely busy, busy, somewhat busy, not busy) 

Erratic work hours (yes, no) 

Self-described busyness (yes, no) 

Self-described exhaustion (yes, no) 


Community 
engagement 


Sense of agency! (yes, no) 
Sense of belonging? (yes, no) 
Voting (yes, no, cannot vote) 


Saliency of 
education 


Privacy or 
security 


Relevance of children’s education (yes, no) 

Importance of education (yes, no) 

Items suggesting privacy or security concerns (such as “no trespassing” signs) seen during 
address observation (yes, no) 

Security of personal information (not concerned, somewhat concerned, highly concerned) 
Access to personal information (information freely available, information somewhat 
available, information protected) 

Who has access to personal information (businesses, social media, government) 
Self-described private person (yes, no) 

Experience with scams (yes, no) 


Government 


Patriotic items seen at residence during address observation (yes, no) 
Attitudes toward federal government (positive, neutral, negative, mixed) 
Attitudes toward federal government data collection (positive, neutral, negative, mixed) 


Mail 
processing 


Mail access type seen during address observation (attached to home, at end of 
driveway/across street/end of road, box/slot/room in multi-unit building, none in view, could 
not determine) 

Mail checking routine (daily, weekly, monthly) 

Mail sorting behaviors (break, no break; important first, junk first; open most, open few; 
discard quickly, keep at residence) 

Challenges with mail delivery (yes, no) 


Surveys 


Attitudes toward surveys (open, neutral, mixed, negative) 
Willingness to participate in surveys (yes/no/depends) 
Survey fatigue (yes/no) 


NHES:2019 
mailings 


Which mailings remembered (for each mailing: yes, no) 
Remembered at least one mailing (yes, no) 

Opened at least one mailing (yes, no) 

Reaction after opening (completed survey, saved for later, discarded) 


1 Participants were considered to have a sense of agency if they felt their actions could make a difference. 
2 Participants were considered to have a sense of belonging if they noted that they felt close to a particular group or community. 


We grouped the 15 participants whose household responded to the NHES prior to 
participating in the interview into a “Late Respondents” group (Group 1). In placing the rest 
of the participants into typology groups, our primary goals were to create groups (1) that 
best represented common themes in the factors that drove nonresponse, (2) whose 
members shared common drivers of nonresponse, and (3) whose reasons for nonresponse 
were the most salient in that group. Through multiple reads of the data, we created six 
additional groups that aimed to meet these goals, and the remaining 70 participants were 
sorted into the group that best corresponded to their attitudes, opinions, and behaviors. Each 
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participant was a member of one—and only one—group. Although some participants 
expressed attitudes, opinions, and behaviors that spanned multiple groups, they were 
ultimately placed in the group that best explained why they did not respond to the survey. 


6.2 Typology Group Characteristics 


Exhibit 6.2 provides an overview of the defining features of each of the seven groups and the 
number of participants in each group. 


Exhibit 6.2. Overview of typology groups 


Typology group Number of Description of group 


participants 
Late Respondents 15 These participants lived in households that ended up responding to the 
(Group 1) NHES prior to participating in the qualitative nonresponse study (in 
response to the fourth screener package). 
Not Enough Time 16 These participants displayed an openness toward surveys and research in 
(Group 2) general but faced time constraints around completing the NHES in 
particular. 
Negative Attitudes | 13 These participants also displayed an openness toward surveys and 
Toward the research in general but held negative views about the federal government 
Federal that made them less willing to participate in a government-sponsored 
Government survey. Over half of the group members described themselves as private, 
(Group 3) the highest rate of any group. 
Federal 13 These participants believed that the federal government already has 
Government access to their information; thus, they felt that completing a government- 
Already Has My sponsored survey was not important or necessary. Some of them were 
Information very concerned about a lack of privacy around personal information and 
(Group 4) for others it was a fact of life. 
Not Relevant to Me | 11 These participants thought that the NHES was not relevant to them 
(Group 5) because they did not have school-age children in the home. They also 


tended not to be open to survey requests in general, but that reluctance 
was not linked to attitudes about the government. 


Multiple Barriers 10 These participants tended to note multiple attitudinal and or lifestyle 
(Group 6) barriers to completing the survey—from time constraints to negative 
opinions about surveys or about the government to concerns about 
privacy. Given their multiple sources of reluctance to participate, they 
would likely be the hardest group to convert to being respondents. 


Less Likely to 7 These participants did not have clear attitudinal or lifestyle barriers to 
Recall NHES completing the survey; they did not tend to have extreme demands on 
Mailings (Group 7) their time or negative opinions about surveys, privacy, or the 


government. However, they were among the least likely to recall the 
NHES mailings. They were likely to report mail delivery issues and to live 
in households with a larger number of adults, which may help explain 
why they did not remember the NHES mailings. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Surveys Program (NHES), 2019. 


Each of the following sections provides additional detail about the characteristics of the 
seven typology groups. Exhibit 6.3 at the end of this chapter also provides a summary of the 
key characteristics of each group relative to the other groups. 
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e First, we discuss the characteristics of each group in terms of the behavioral and 
attitudinal factors that were used to create the groups. 


e Next, using address observations and self-reports collected during the interviews, we 
summarize the characteristics of each group to provide additional context about its 
members (see tables 3.1, A.3.4, and A.3.5 for the distribution of these variables across 
all 85 participants).26 


e Finally, we look at the characteristics of each group in terms of the address- and area- 
level auxiliary data available on or appended to the NHES sampling frame (see tables 
A.3.2 and A.3.3 in appendix A for the distribution of these variables across all 85 
participants). The purpose of this analysis was to determine whether there are 
variables on the sampling frame that are associated with membership in a particular 
typology group. Although some of the frame variables have high item-missing rates 
and do not consistently align with participants’ self-reported characteristics (see 
chapter 7 for more details), this is the only information that is available about 
sampled addresses prior to the survey administration—and thus the only information 
available to use to for targeted mailing of survey materials that address concerns that 
are only relevant to certain typology groups. 


In the group descriptions, we discuss only the defining characteristics of each group. By 
focusing on what makes them unique, we can better understand which factors matter when 
it comes to not responding—or responding late—to the NHES. If a particular factor or 
characteristic is not mentioned for a group, it can be assumed that the group was similar to 
the other groups for that factor or characteristic. Due to the small number of interview 
participants, we focused on general patterns and did not conduct statistical testing. 


6.2.1 Group 1: Late Respondents (N = 15) 


As noted above, Group 1 consisted of “late respondents’-everyone in Group 1 lived in a 
household that responded to the NHES after the fourth screener package. Most of these 
households responded to the screener by paper, with a few responding by web or inbound 
telephone. The late respondent group shared characteristics across numerous dimensions 
that set them apart from participants that lived in households that did not end up responding 
to the NHES (“final nonrespondents”). 


Group 1 participants held positive or neutral attitudes toward surveys and research. In 
general, they were open or moderately open to doing different types of surveys. No one 
discussed having survey fatigue. They had positive or mixed reactions to government- 
sponsored surveys and tended to point out the benefits of survey participation. They 
expressed more enthusiasm about the Census than did final nonrespondents, more often 


26 We considered the following participant and household characteristics that were self-reported by participants during the interviews: 
the participant’s age, gender, race/ethnicity, educational attainment, enrollment status, and employment status; the number of adults in 
the household; whether children live in the household; whether the household is Spanish speaking; household income; and type of 
internet access at home. We also considered the following characteristics that were collected as part of the address observations: 
structure type; the presence of children; internet or television connectivity; signs of outdoor living; and other outdoor decor. 
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sharing positive views and their intention to complete the 2020 Decennial Census. 
Additionally, almost all of them either explicitly discussed the importance of education 
during the interviews or were employed in the education field. 


They appeared to be less concerned than final nonrespondents about privacy. Most late 
respondents explicitly said they were not very concerned about privacy,as compared 
to only one in three final nonrespondents. In addition, late respondents were less likely to 
self-identify as a very private person. While privacy and data security mattered to the 
participants in this group, unlike the participants in some of the other groups, they did not 
feel as if their data would be compromised by completing the NHES screener. Importantly, 
no one in this group enacted extreme measures to protect their privacy, such as burning their 
mail, or believed that the government already had access to all their personal data. Although 
some late respondents expressed concerns during the interview about sharing information 
about their children in a survey, most of the late respondents sampled for a topical survey 
completed it. This group also had one of the highest percentages of observed privacy or 
security concerns indicators, but this prevalence may be related to the group’s unique 
demographic characteristics. First, this was the only typology group where no one lived in 
an apartment, which enabled observers to access these units more consistently. This group 
also tended to report higher household income, which could suggest that features such as 
security cameras were more related to protecting home assets than privacy concerns. 


All participants in this group felt like their voice could make a difference in some way, 
whether through voting or participating in a group or community activity. While this sense 
of agency is not unique to this group, it is notably absent in other groups and helps explain 
why responding to a survey is concurrent with Group 1’s worldview that individual action 
can influence larger issues. Having a sense of belonging did not overtly arise in many 
interviews with late respondents; however, when it did, it tended to be discussed in terms of 
feeling connected to ethnic or immigrant communities. 


Similar to other groups, late respondents tended to have mixed or negative opinions about 
the federal government. However, unlike some of the other groups (namely, Groups 3 and 
4), these opinions did not prevent them from responding to the NHES. They separated the 
need for federal agencies to collect data from their concerns about the government in general 
or the current political situation. They were also the only group where no one shared 
negative opinions about the Department of Education, which may suggest that they do not 
view the federal government as a singular, monolithic entity. 


Late respondents tended to live in households that did not have a break between mail 
retrieval and mail sorting. They also tended not to open gray-zone or junk mail and disposed 
of mail quickly. These differences in mail processing suggest that late respondents routinely 
categorized NHES mailings differently than final nonrespondents did. During the interviews, 
all late respondents believed that the NHES mailings looked important and official, which 
suggests that the NHES mailings did not fall into the gray zone for them. They were more 
likely than those in several other groups to access their mail via a mail slot or mailbox 
attached to their home. 
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What kept Group 1 participants from responding to the NHES earlier seems to be related to 
the demands on their time. While many study participants reported being busy, late 
respondents discussed having extreme time pressures more frequently than most other 
typology groups (Group 2 was the exception). 


Participant characteristics collected during qualitative nonresponse study 


Of all groups, this group had the highest rate with a bachelor’s degree or higher. They had 
the most households with income over $100,000 of any typology group. They were also more 
likely than the participants in most other groups to have two adults in the home. They all 
reported having home internet access via a computer—again, the highest rate of any 
typology group. Nearly all the participants in this group were observed to be living in single- 
family homes. They also were observed to have other outdoor decor more often than almost 
all other groups. 


Participant characteristics available on sampling frame 


Similar to self-reported characteristics collected during the interview, participants in this 
group were more likely than participants in other groups to have a head of household with 
a bachelor’s degree or higher and to have a household income of $75,000 or more. Consistent 
with the address observations, they were more likely to live in single-unit dwellings than in 
multi-unit ones, and they were more likely to live at addresses that were owner occupied. 
This group also had the highest percentage of participants whose household was flagged on 
the frame as having children. All of them had a phone number available on the frame, the 
highest rate of any typology group. They were also less likely than most other groups to be 
missing information about the age of the head of household, but this pattern was not 
repeated for some of the other head of household characteristic variables available on the 
frame (such as gender or race/ethnicity). 


6.2.2 Group 2: Not Enough Time (N = 16) 


While busyness seemed to impede Group 1 participants from responding to the NHES prior 
to the fourth screener package, it was an insurmountable barrier for those in Group 2. 
Everyone in this group described themselves as being extremely busy, with some also having 
erratic schedules due to work. Many talked about there being extreme demands on their 
time, some exclusively from work obligations and others from a mix of work and family 
constraints. Some discussed being completely exhausted. Group 2 members held a variety of 
jobs—from the service sector to the executive level—with the common thread being that 
they needed to put in long hours or travel extensively for their job. They discussed waking 
early and going to bed late, commuting for several hours every day or juggling child care. 
This group also had the highest percentage of participants who were both currently enrolled 
as students and employed for pay. Although homeschooling was reported infrequently by 
participants, all those who did so were in this group.?” 


27 Two participants were currently homeschooling at least one child, and one additional participant had previously homeschooled at least 
one child (who was now an adult). 
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In general, Group 2 had relatively open or positive attitudes toward surveys. Of those who 
talked about the federal government and research, almost all expressed positive views. 
Despite this support for federal research, Group 2 participants expressed varying opinions 
about the federal government itself, with most being neutral or negative. Over half of the 
participants in this group said that they voted, the highest of all typology groups. Those in 
Group 2 tended to have some concerns about data security, particularly around social media, 
but generally were not extremely concerned; this group had one of the lowest percentages 
of participants for whom evidence of privacy or security concerns were noted by observers. 
Very few described themselves as a private person. While almost everyone in this group felt 
a sense of agency, not many explicitly talked about a sense of belonging to a community. 
Instead, many noted that they wanted to participate in different activities—such as 
volunteering—or spend more time with others, but time commitments made it impossible. 
While the majority did not talk about a sense of belonging, of those who did, all expressed 
feeling connected to their community via group activities. 


In general, Group 2 participants checked their mail frequently and most let mail sit in the 
household longer than one day after sorting. This group had the highest proportion of 
participants who took a break between checking and sorting mail. They were somewhat less 
likely to access their mail via a mail slot or mailbox attached to their home. Of those who 
opened an NHES mailing, several explicitly said that they did not have time to take surveys. 
One participant who works from 11:00 p.m. to 7:30 a.m. explained that she opened and read 
one of the letters after receiving it but did not respond because she was on her way to bed 
(7057). 


Participant characteristics collected during qualitative nonresponse study 


Group 2 participants had higher proportions of participants who were Black, were female, 
and had children in the household than several other groups. It had one of the highest 
percentages of the presence of children noted during the observations. Group 2 participants 
were also less likely than participants in several other groups to be 55 years or older. As 
noted earlier, this group had the highest percentage of participants who were both employed 
for pay and enrolled in school. About half of the participants were the only adult in their 
household. Their household income varied, but none reported household income over 
$100,000. This was one of only two groups where a participant reported not having internet 
access at home. 


Participant characteristics available on sampling frame 


Similar to the self-reported characteristics collected during the interview, none of the 
participants had a household income of $100,000 or more, but contrary to the self-reports, 
they were less likely to be flagged as having children. None of them had a head of household 
with a bachelor’s degree or higher. Participants in this group also had higher rates of missing 
information for several of the frame variables than participants in other groups. 
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6.2.3 Group 3: Negative Attitudes Toward the Federal Government (N = 13) 


All Group 3 participants held negative attitudes toward the federal government. Their 
concerns included believing the federal government was intrusive, lacking confidence in 
current leadership, and fearing deportation or other negative sanctions at the hands of the 
government. While very few participants overall had positive perceptions of the 
government, members of this group tended to be particularly suspicious or wary of it. 


Group 3 members’ opinions about the government fell into one of two groups. About half 
wanted as little interaction with the government as possible so as not to jeopardize their 
current situation. For some, the fear was rooted in their immigration status. For others, they 
believed their past incarceration history made them vulnerable. The other half of Group 3 
deeply believed that the government was generally corrupt and could not be trusted. These 
participants talked about the government specifically and purposefully creating barriers for 
women and minorities. Only one member of this group had observable signs of patriotism 
outside of the residence. In general, the participants in this group who discussed voting 
either did not do so or did not believe their votes always counted. 


Group 3 participants actively protected their data in some way. Over half of the group 
members described themselves as private, the highest of any typology group. This was the 
only group where all members were recruited via in-person visits. But almost no one in this 
group stated that they believed the government already had access to all their data. Although 
they were reluctant to participate in a government data collection, almost everyone in this 
group had positive attitudes about nongovernmental surveys and was open to completing 
them. 


While some members also talked being busy or having erratic work schedules, they, unlike 
Group 2, did not talk about being exhausted or overwhelmed with daily responsibilities. Most 
Group 3 participants felt a sense of agency—lower than in the first two groups but higher 
than in Groups 4, 5, and 6—and most did not discuss a sense of belonging. 


Most Group 3 participants did not say how frequently they checked their mail. Of those who 
did, several did not check it very often. These group members tended to sort mail 
immediately after it was retrieved and rarely opened gray-zone or junk mail. This group had 
one of the lowest proportions of members who remembered receiving at least one NHES 
mailing; however, all Group 3 participants who remembered a mailing opened it. That said, 
upon opening the mailing, this group had the highest percentage of participants who then 
outright rejected the survey request (as opposed to saving it for later). Several either were 
rattled to receive any type of governmental correspondence or they dismissed it outright 
because they did not trust or want to interact with anything related to the government. 
Compared to the other typology groups, they were the most likely to not have a mailbox or 
slot in view during the observations. 


Participant characteristics collected during qualitative nonresponse study 


The participants in this group had a relatively higher proportion of members whose highest 
education was a high school degree or less. This group also had one of the highest 
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percentages of people who refused to share their race or ethnicity. They were more likely 
than most other groups to be observed to live in a single-unit structure. Finally, they were 
more likely than participants in most other groups to have two adults in the household. 


Participant characteristics available on sampling frame 


Very few Group 3 members lived in the West. None had a head of household who was age 55 
or older or had three or more adults in the household. This typology group was also more 
likely to be missing information for several of the frame variables. In contrast to the self- 
reports, frame information indicated that about half of households this group had a White 
head of household. 


6.2.4 Group 4: Federal Government Already Has My Information (N = 13) 


None of the Group 4 participants saw the purpose of or the urgency in completing the NHES, 
as they believed that the federal government already had access to their data. Whether it was 
through other federal initiatives such as taxes or the Census, through data selling by 
marketing firms, or via hackers, this group believed that their data were freely available and 
that the government should make use of that before asking them to complete a survey. 


Group 4 members held one of two positions about the government—and other entities— 
having access to their data. They were either extremely concerned about the volume of 
personal data to which they believed the government had access, or they were not very 
concerned because the volume of data available suggested that there was little they could 
consistently do to protect their information. 


While Groups 2 and 3 were generally open to nongovernmental surveys, Group 4 participants 
tended not to be open to those surveys either. Their opinions about the federal government 
also were more mixed than the negative attitudes seen in Group 3. Group 4 talked about 
being busy, but they did not discuss having erratic work schedules or high levels of 
exhaustion as frequently as Group 2. About half of Group 4 participants did not feel a sense 
of agency, lower than in Groups 1-3. Some felt a sense of belonging, one of the highest rates 
in any typology group, mainly through being connected to their neighborhood or religion. 


This group tended to sort mail immediately after checking and discard unwanted mail 
quickly. This group had the highest percentage of participants who were observed to access 
their mail via a mailbox, slot, or room in a multi-unit building, and it was the only group 
where none of the address observations concluded that there was not a mailbox or slot in 
view. 


Participant characteristics collected during qualitative nonresponse study 


Group 4 participants tended to be younger, with almost half being under 35 years old. This 
was the only group where the majority of participants were White, and all participants spoke 
English most often at home. They were less likely than the other groups to be Hispanic and 
somewhat more likely than several other groups to be female. They were also the most likely 
to refuse to give their annual household income. This group had one of the lowest 
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percentages of participants who lived in households with three or more adults and one of 
the highest percentages of participants who were observed to live in apartments. They were 
among the groups with the highest observed rates of internet or television connectivity and 
other outdoor decor. 


Participant characteristics available on sampling frame 


Members of this typology group were more likely to live in the Northeast region and less 
likely to live in the South. Similar to the self-reported characteristics, this group was more 
likely to have a White head of household. They also were more likely to have a head of 
household who did not have a high school credential. Group 4 members were more likely to 
have a phone number available on the frame and less likely to be missing frame information 
for several of the variables. 


6.2.5 Group 5: Not Relevant to Me (N = 11) 


Everyone in Group 5 believed that they did not need to complete the NHES because K-12 
education was not relevant to their household. Only one household had a child living in it, 
and this was the participant’s preschool-age granddaughter. The participant did not believe 
the NHES applied to her because she was not the child’s mother and the child was not in 
elementary school yet (6331). 


Compared to other groups, Group 5 participants had the fewest number of people who 
discussed being extremely busy or having erratic work schedules. No one discussed being 
exhausted. They were more likely than other groups to be employed in manual labor. In 
terms of mail sorting, this was the only group where everyone held onto their mail and did 
not discard it quickly. 


Overall, most Group 5 participants said they were very open to completing surveys, yet this 
group also had the highest proportion of participants reporting survey fatigue. They had a 
wide range of privacy and security concerns, ranging from none at all to very extreme. No 
one discussed the government or other entities having access to their personal information. 
Unlike other groups, the members of Group 5 did not often express their opinions about the 
federal government—either about the government as a whole or about specific topics, like 
federal data collections. About half expressed a sense of agency, similar to Group 4. Most did 
not discuss a sense of belonging, but those who did felt connected through shared group 
activities. 


Of the groups, Group 5 had the highest proportion who sorted unimportant mail first. Among 
those who remembered at least one NHES mailing, Group 5 had a lower rate who opened 
them than in most other groups. Among those who opened at least one mailing, this group 
also had the highest percentage of participants who saved the mailing for later, although it 
was usually to give the materials to a household with children. Group 5 members were the 
most likely to be observed to access their mail via a mail slot or mailbox attached to their 
home. Finally, for almost all the household attribute observations, this group had the lowest 
percentage of participants who were observed to have evidence of those attributes. 
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Participant characteristics collected during qualitative nonresponse study 


Almost all the participants in this group were people of color; in particular, nearly half of 
them were Black. Very few were under age 35. Compared to the other nonrespondent 
groups, Group 5 participants generally had higher household income. As noted above, almost 
none of them had children living in the home. Finally, this group had the highest percentage 
of participants who only had home internet access via a phone or tablet (and not via a 
computer). 


Participant characteristics available on sampling frame 


The majority of the group members lived in an urban area, one of the highest rates of any 
group. No one lived in the Northeast region. Similar to the self-reported characteristics, all 
had a head of household who was between 35 and 64 years old, and they were more likely 
to have a Black head of household. They were also less likely to have a male head of 
household and more likely to have a head of household with some college but no bachelor’s 
degree. They were more likely to be part of single-adult households and to be renting their 
home. Contrary to the self-reports, none had a household income of $100,000 or more. 


6.2.6 Group 6: Multiple Barriers (N = 10) 


Participants in this group reported experiencing multiple barriers to completing the NHES. 
These participants would likely be the hardest to convert to respondents. Unlike the other 
groups, where one driver of nonresponse appeared particularly salient, Group 6 members 
each had a combination of factors that seemed to influence their lack of response. 


Almost all Group 6 participants discussed having very little time due to work or family 
obligations. Many also discussed having erratic schedules. About one in three talked about 
being exhausted. Group 6 participants tended to hold negative opinions toward surveys and 
research and not to be open to participating; one noted that sharing personal information 
“could come back to hurt you.” (7177) Of those who shared an opinion about survey modes, 
none of them liked taking surveys online, the most of any typology group. Group 6 
participants also tended to have negative views on the federal government conducting 
research. 


Most believed that the NHES’s focus on children and education made it not relevant to them 
personally since they either had no school-age children in the home or the children who were 
living there were not their children. Over half reported challenges with mail delivery. Group 
6 did have one of the highest rates of recall of the NHES mailings, but most of the participants 
in this group who opened the mailings ended up rejecting the survey request outright (as 
opposed to saving the materials for later). 


Group 6 had the highest proportion of members who were very concerned about privacy and 
data security. Several members of this group asked for their interview not to be recorded 
specifically because of privacy concerns. This group also had one of the highest rates of 
observed indicators of privacy or security concerns. Group 6 participants held a variety of 
negative attitudes toward the federal government, including distrusting it in general, fearing 
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that government is too intrusive, and thinking that money has too much influence in politics. 
Very few people in this group felt a sense of agency, the lowest rate in all the groups, and 
most group participants did not talk about a sense of belonging. 


Participant characteristics collected during qualitative nonresponse study 


Group 6 was the only one where the majority of participants were male. Participants in this 
group were somewhat more likely than those in several other groups to be Hispanic; this 
group also had one of the highest rates of participants who spoke Spanish most often at 
home. This was one of only two groups (the other being Group 2) where a participant 
reported not having internet access at home. Participants in this group were among the most 
likely to be observed to have indicators of valuing outdoor living. 


Participant characteristics available on sampling frame 


Members of this typology group were more likely to live in the Midwest than in other regions. 
They were also one of the groups that was less likely to have a phone number available on 
the frame. Contrary to the self-reported characteristics, they were less likely than most other 
groups to have a Hispanic head of household. 


6.2.7 Group 7: Less Likely to Recall NHES Mailings (N = 7) 


Everyone in Group 7 checked their mail frequently, yet just under half remembered at least 
one NHES mailing and almost no one in the group remembered the second or fourth screener 
packages. This group also had the highest percentage of participants who said they tend to 
open most of the mail they receive; however, among those who remembered at least one 
NHES mailing, this group was the least likely to report opening one. About half talked about 
challenges with mail delivery. 


Other than their lower rate of recall and higher rate of mail delivery issues, the other defining 
characteristic of participants in this group is that they did not have strong opinions or 
extreme life experiences for the other factors used to create the typology groups. More than 
half of Group 7 participants described themselves as being busy, but no one spoke of erratic 
schedules or exhaustion. Most group members had no concerns or only mild concerns about 
data security, as well as being relatively open to participating in surveys They did not talk 
often about how they felt about the federal government in general or about federal data 
collections. Of those who did, their opinions were positive or mixed. Everyone in this group 
expressed a sense of agency and in general did not talk about a sense of belonging. 


Participant characteristics collected during qualitative nonresponse study 


Almost all the participants in this group were female. This was the only typology group 
where no one had a bachelor’s degree. It had one of the highest proportions of Hispanic 
participants, as well as the highest percentage who spoke Spanish most often at home. It also 
was the least affluent of all typology groups, with over half having a household income of 
$30,000 or less. This group had the highest percentage of participants who lived in 
households with three adults or more; this may be one explanation for why they were less 
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likely to remember the NHES mailings (e.g., if another adult retrieved the mail on the days 
the letters arrived). Participants in this group were more likely than several other groups to 
be observed to live in apartments. They had one of the highest rates of observed internet or 
television connectivity. 


Participant characteristics available on sampling frame 


In general, the frame characteristics for Group 7 mirrored many of the self-reported 
characteristics and observed items. This group was more likely to have a Hispanic head of 
household; none had a head of household with a bachelor’s degree, and none had a 
household income of $75,000 or more. They were also more likely to have a household 
comprising three adults or more, to live in a multi-unit dwelling, and to be renters. Finally, 
Group 7 was less likely than most other groups to be flagged as having children and, for 
almost all the frame variables, less likely to be missing information. 
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Exhibit 6.3. Typology group characteristics 


Characteristics available on NHES 
sampling frame 


Attitudes Mail behaviors and NHES mailing 


behaviors 


Characteristics collected during 
qualitative nonresponse study 


Typology 


group 


Group 1: Late 
Respondents 


Likely to be open to surveys and 
research 

Less likely to mention being 
concerned about privacy 

Likely to believe their actions make 
a difference 

Likely to be extremely busy and 
most likely to have erratic schedules 


Tend not to break between mail 
retrieval and mail sorting 

Tend not to not open gray-zone or 
junk mail 

Tend to dispose of mail quickly 
Most likely for those who remember 
at least one NHES mailing to open it 
Completed NHES after the fourth 
screener package 


Higher educational attainment 
Higher household income 

All have internet access via 
computer at home 

No apartment dwellers 

More likely to access mail via mail 
slot or box attached to home 
More likely to have 2 adults in 
household 


Higher head of household 
educational attainment 

Higher household income 

More likely to be flagged as having 
children 

More likely to live in single-unit 
dwellings 

More likely to be owner occupied 
All have phone number available 


Group 2: Not 
Enough Time 


Likely to be open to surveys and 
research 

Most likely to talk about exhaustion 
and multiple work and home 
responsibilities 

Most likely to be extremely busy 
Likely to believe their actions make 
a difference 


Tend to break between mail retrieval 
and mail sorting 

Tend to keep mail longer than one 
day 


NHES mailing 

Most likely for those who remember 
at least one NHES mailing to open it 
More likely for those who open at 
least one NHES mailing to discard it 


More likely to be Black 

Likely to be female 

Less likely to be 55 or older 

More likely to have children in the 
household 

Highest percentage both employed 
for pay and enrolled in school 


Most likely not to have a mailbox or 
slot in view 

More likely to have 2 adults in 
household 


None have head of household with 
bachelor’s degree 

Less likely to be flagged as having 
children 

None have household income of 
$100,00 or more 

For several frame variables, more 
likely to be missing information 


Group 3: All very distrustful of the federal - Tend not to break between mail Most racially/ethnically diverse - None with head of household age 55 
Negative government retrieval and mail sorting group or older 

Attitudes Likely to be very concerned about - Tend not to open gray-zone or junk More likely to have high school - None have three or more adults 
Toward the privacy mail degree or less - For several frame variable, more 
Federal - Less likely to remember at least one Likely to live in single-unit structure likely to be missing information 
Government 


Very few live in West 
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Typology 


group 


Attitudes 


Mail behaviors and NHES mailing 
behaviors 


Characteristics collected during 
qualitative nonresponse study 


Chapter 6. Nonrespondent Typologies 


Characteristics available on NHES 
sampling frame 


Group 4: - Not likely to be open to surveys or - Tend not to break between mail - More likely to be age 34 or younger - More likely to have White head of 
Federal research retrieval and mail sorting - More likely to be White household 
Government - More likely to believe government has |- Tend to dispose of mail quickly - Less likely to be Hispanic - More likely to have head of household 
Already Has access to their data - More likely to access mail via - All speak English at home who does not have high school 
My . mailbox/slot/room in multi-unit - More likely to live in apartments credential 
Information building - Most likely to refuse to provide - More likely to have phone number 
income available 
- For several frame variables, less likely 
to be missing information 
- More likely to live in Northeast, less 
likely to live in South 
Group 5: Not - Likely to be open to surveys but high - Tend to pull out unimportant mail - Likely to be age 35 or older - Allhave head of household between 
Relevant to survey fatigue during the sorting process - More likely to be Black or Hispanic 35 and 64 years old 
Me - More likely to think children’s - Most likely to keep mail longer than - Less likely to have children in - Less likely to have male head of 
education not relevant one day household household 
- Less likely to report being extremely - Less likely for those who remember at |- Few have householdincome less than |- More likely to have head of household 
busy least one NHES mailing to open it $30,000 with some college but no bachelor’s 
- Not likely to discuss federal - More likely to save NHES mailing after |- Most likely to only have home internet degree 
government opening access via phone or table - More likely to have Black head of 
household 
None have household income of 
$100,00 or more 
More likely to be single-adult 
households 
More likely to be renters 
More likely to live in urban areas 
None live in Northeast 
Group 6: - Least likely to believe their actions - More likely to remember at least one - More likely to be male Less likely to have Hispanic head of 
Multiple make a difference NHES mailing - Somewhat more likely to be Hispanic household 
Barriers - Likely to be very concerned about - More likely to speak Spanish at home Somewhat less likely to have phone 


privacy 


- Likely to be distrustful of federal 


government 


- Likely to be busy and have erratic 


schedules 


- Not likely to be open to surveys 


number available 
Somewhat more likely to live in 
Midwest 
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Exhibit 6.3. Typology group characteristics—Continued 


Typology 
group 


Attitudes 


Mail behaviors and NHES 
mailing behaviors 


Characteristics collected during 
qualitative nonresponse study 


Chapter 6. Nonrespondent Typologies 


Characteristics available on 
NHES sampling frame 


Group 7: 
Less Likely 
to Recall 
NHES 
Mailings 


Likely to believe their actions 
make a difference 

Likely to discuss having 
challenges with mail delivery 
Likely to be open to surveys 
Not likely to discuss federal 
government 


- Tend to open gray-zone or junk 
mail 

- Less likely to remember at least 
one NHES mailing 

- Less likely for those who 
remember at least one NHES 
mailing to open it 

- Less likely for those who open at 
least one NHES mailing to 
discard and more likely to save 


- More likely to be female 

- None have a bachelor’s degree 

- More likely to be Hispanic 

- More likely to speak Spanish at 

home 

- Lower income 

- More likely to live in apartments 

- More likely to have at least 3 
adults in household 


- None have head of household 
with bachelor’s degree 

- More likely to have Hispanic 
head of household, none have 
White head of household 

- None have household income of 
$75,000 or more 

- Less likely to be flagged as 
having children 

- More likely to have 3 adults or 
more 

- More likely to live in multi-unit 
dwellings 

- More likely to be renters 

- Less likely to be missing frame 
information 


SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Surveys Program (NHES), 2019. 
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Chapter 7. Quality of the Sampling Frame 


This chapter presents a series of analyses aimed at better understanding the quality of the 
NHES:2019 address-based sampling frame—and whether the quality of the frame might be 
a driver of nonresponse. In the first section of this chapter, we assess whether some of the 
addresses on the frame should not have been included (e.g., do not exist, nonresidential, 
vacant). In the second section, we explore the characteristics of the addresses that ended up 
with undeliverable as addressed (UAA) outcomes. In the final section of the chapter, we 
examine the quality of the auxiliary variables that are included on the frame. Most of the 
analyses presented in this chapter focus on the results of the address observation component 
of the study. 


7.1 Addresses That Should Not Have Been Included on the Frame 


The target population for NHES:2019 was all residential addresses in the United States; all 
U.S. civilian, noninstitutional, residential addresses were eligible to be sampled. Using the 
results of the observation component of the study, this first section of the chapter 
investigates whether some of the addresses on the NHES sampling frame should not have 
been included, because they appear to not exist, to not be occupied, or to not be residential. 
This evaluation is important because these types of addresses cannot respond to the NHES. 
Their presence in the sample would thus have a negative effect on the survey response rate. 
In addition to presenting the distribution of responses to observation items, we summarize 
the notes provided in write-in items on the observation form and in the case management 
system that was used to track fieldwork efforts. 


The analyses presented in this section exclude addresses that ended up responding to 
NHES:2019 after the fourth screener package and addresses that had UAA outcomes (23 
percent of all addresses in the study sample). Findings were expected to differ for addresses 
that had UAA outcomes; therefore, these addresses were analyzed separately and were 
compared with non-UAA addresses (see section 7.2). 


7.1.1 Observability 


As shown in figure 7.1, observers were able to conduct at least a partial observation for 93 
percent of the addresses and a full observation for 76 percent of them (see also table A.7.1 
in appendix A). Partial observations were those observations where the observer was able 
to observe the exterior of the building, but not the interior (including the entrance to the 
sampled unit itself). This tended to occur due to access constraints at multi-unit structures, 
such as needing a key, code, or access card to enter the building. Although observers 
requested entry to multi-unit buildings via the front desks or security personnel whenever 
possible, many either were refused entry or simply could not find a staff member to grant 
them access. 


Observers were unable to observe about 7 percent of the addresses for various reasons, 
including not being able to reach the address due to access constraints in places like gated 
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communities or apartment complexes; not being able to locate the address (e.g., no 
apartment in the building with the correct unit number, no house on the street with the 
correct house number); or another reason (e.g., being down a long private driveway or in an 
area too unsafe for the observer to conduct an observation). 


Figure 7.1. Percentage distribution of observation outcomes for nonrespondent 
addresses sampled for observation component: 2019 


Percent 
100 


80 76 
60 
40 


20 


Observed Partially observed * Not observed, could Notobserved, could Not observed, other 
not access address? _ not locate address? reason 4 


Observation outcome 


Partially observed addresses are those where the observer was able to observe the exterior of the multi-unit building but was not able to enter the building 
to observe the interior entry to the sampled unit. 

2Addresses that could not be accessed are those where observers could not get close enough to confirm whether the address existed (for example, if the 
address was in a gated community to which the observer could not gain access). 

3Addresses that could not be located are those where the observer was able to access the location where the address should have been, but the observer 
could not find any evidence that the address existed. 

4Addresses that were not observed for another reason were those where the observer could not make the observation for reasons other than not being able 
to access or locate the address (for example, because the observer was concerned about safety). 

NOTE: Addresses that ended up responding to NHES:2019 and addresses that had at least one undeliverable as addressed (UAA) NHES:2019 mailing were 
excluded from this analysis. Details may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Surveys Program (NHES), 2019. 


We also assessed whether selected address characteristics were significantly associated 
with observability. These analyses were conducted after collapsing the three observation 
fieldwork outcomes that led to the address not being observed (could not access address, 
could not locate address, and not observed for another reason) to a single “not observed” 
category. Hence, the outcome variable for these analyses used a collapsed version of 
observability that had three categories: observed, partially observed, and not observed. 
Wald joint significance tests indicated that 7 of the 21 independent variables included in the 
multivariate logistic regression model were significant predictors of observability. Exhibit 
7.1 lists the 7 significant predictors of observability and notes which address types were 
more likely to be fully observed (also see tables A.7.2 and A.7.3 in appendix A). Subgroup 
variation in observability tended to be driven by differences in the prevalence of fully 
observed and partially observed outcomes—and was rarely driven by differences in the 
prevalence of “not observed” outcomes. For example, addresses with a “high rise” route type 
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were more likely than those with a “street” route type to only be partially observed (40 
percent versus 3 percent), likely due to the access constraint issues discussed above. 


Exhibit 7.1. Significant predictors of observability in multivariate logistic regression 


Independent variable Details 


Phone number 
available on the 
sampling frame 


Addresses with phone number more likely than those without phone number to 
be fully observed (83 percent versus 66 percent) 


Tract poverty rate 


Percentage of 
households in Census 


Route type Street addresses more likely than high rise addresses to be fully observed (91 
percent versus 51 percent) 

Dwelling type Single-unit addresses more likely than multi-unit addresses to be fully observed 
(93 percent versus 53 percent) 

Urbanicity Suburban addresses more likely than urban addresses to be fully observed (84 
percent versus 68 percent) 

Region Northeast and South addresses somewhat more likely than Midwest and West 


addresses to be fully observed (e.g., 79 percent of Northeast addresses versus 72 
percent of West addresses) 

Addresses in high-poverty tracts! somewhat less likely than addresses not in 
high-poverty tracts to be fully observed (72 percent versus 78 percent) 
Addresses with a higher percentage of households with children in the block 
group more likely than those with a lower percentage of households with 


block that include a children to be fully observed (82 percent of fourth quartile versus 64 percent of 
child first quartile) 


1High poverty tracts are those where 20 percent or more of the addresses were below the poverty level. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Surveys Program (NHES), 2019; U.S. 
Department of Commerce, American Community Survey (ACS), 2013-2017 and Decennial Census, 2010; and Federal Communications Commission (FCC), 
2017. 


7.1.2 Residential Occupancy Status 


As shown in figure 7.2, observers determined that 73 percent of the observed addresses 
were occupied residential units (see also table A.7.4 in appendix A). Examples of commonly 
cited indicators of current occupancy included cars in the driveway, doormats, lights on in 
the home, trash cans or recycling bins in the yard or driveway, patio furniture, decorative 
items on the door or in the yard, pets in the yard or sitting in a window, potted plants or well- 
maintained landscaping, and bikes or children’s toys. In a few cases, observers happened to 
see a resident in the yard or entering or exiting the unit. 


For another 26 percent of the observed addresses, observers determined they were 
residential but were not able to determine the occupancy status. This occurred mostly with 
addresses where the observer could only observe the outside of a multi-unit building but did 
not have access to the sampled unit itself to check for the presence of indicators that 
someone was living in the unit. In other cases, observers were able to fully observe the 
address, but there were neither indicators of occupancy nor that the address was vacant or 
unlivable (e.g., no cars that could be attributed to the sample unit; no lights on; or no personal 
items that indicated current occupancy). Additionally, some observers identified conflicting 
evidence. 
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About 1 percent of the observed addresses were determined to be vacant residential units, 
including temporarily vacant, seasonally vacant, and permanently vacant addresses. 
Temporarily vacant addresses appeared to be for sale or lease (e.g., “for sale/lease” signs, 
lock boxes on the door. Permanently vacant addresses appeared to be under construction or 
uninhabitable (e.g., boarded up or broken windows). Seasonally vacant units appeared to be 
in a resort area. For the majority of these addresses, observers noted that they were single- 
unit structures. Finally, a very small number of the observed addresses were determined to 
be vacant lots, and none of them were determined to be commercial businesses. Due to the 
small percentage of addresses that were determined to be nonresidential, subgroup analyses 
were not conducted for this observation outcome. 


Figure 7.2. Percentage distribution of residential occupancy status observation for 
nonrespondent addresses sampled for observation component: 2019 


Percent 
100 


80 73 
60 


40 
26 
20 


1 ¢ # 


Occupied residential Residential unit, could Vacant residential unit’ Vacant lot Commercial business 
unit not determine 
occupancy status 


Residential occupancy status observation 


# Rounds to zero. 

+Reporting standards not met. There are too few cases for a reliable estimate. 

1Vacant residential units include: (1) temporarily vacant addresses where the unit had a for sale or for rent sign or a lock box and there was no evidence of 
current occupants, (2) seasonally vacant addresses where the unit was located in a resort area and was well maintained enough to suggest that it had 
tenants during other parts of the year, but there was no evidence of current occupants, and (3) permanently vacant addresses where the address appeared 
uninhabitable (e.g., boarded windows/doors, holes in the walls or roof), was condemned, or was under construction. 

NOTE: Residential occupancy status was not collected for addresses that were not able to be observed (for example, for addresses that could not be located). 
Addresses that ended up responding to NHES:2019 and addresses that had at least one undeliverable as addressed (UAA) NHES:2019 mailing were excluded 
from this analysis. Detail may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Surveys Program (NHES), 2019. 


7.2 Addresses with Inconsistent NHES:2019 UAA Outcomes 


To better understand the drivers of UAA outcomes, this section discusses the characteristics 
of addresses that had inconsistent NHES:2019 UAA outcomes—that is, addresses that had 
some, but not all, of their NHES screener mailings returned as undeliverable as addressed. 
The purpose of this evaluation is to inform decisions about the viability of continuing to send 
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survey mailings to such addresses and about whether different types of contact approaches 
might be more successful for reaching them. 


Among the 760 addresses sampled for the observation component of the study, there were 
about 85 addresses with inconsistent UAA outcomes for NHES:2019 screener mailings. Of 
these, 60 addresses were sampled for the observation component as inconsistent UAA 
addresses because they had a UAA outcome for at least one (but not all) of the first three 
NHES:2019 screener packages. The rest were identified during field work after the initial 
sampling for the observation study.?8 


Using auxiliary data available via the sampling frame and other publicly available sources, 
we compared the characteristics of the 85 inconsistent UAA addresses to those of the 
nonrespondent addresses that were sampled for the observation component but did not 
have any UAA outcomes (“non-UAA” addresses). Due to the relatively small number of cases 
with inconsistent UAA outcomes, the focus was on general patterns and statistical testing 
was not conducted.?? 


Table 7.1 summarizes these comparisons. For example, inconsistent UAA addresses were 
more likely than non-UAA addresses to be multi-unit addresses. They also had higher rates 
of missing frame data (e.g., household income; phone number) and were less likely to be 
flagged as having children in the household. By contrast, there was relatively little difference 
between inconsistent UAA addresses and non-UAA addresses in the characteristics of the 
head of household or the urbanicity of the address. 


28 See appendix C for discussion of a sensitivity analysis that was conducted to determine whether these cases should be included here. 
29 Readers should be aware of the potential limitation that the sample design for this study may also be a driver of differences between 
the two groups. For example, certain types of addresses were oversampled among non-UAA cases to improve representation of key 
subgroups among interview participants, while this was not done for UAA addresses since they were not eligible for interview 
participation. 
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Table 7.1. Number and percentage distribution of auxiliary data for 
nonrespondent addresses sampled for observation component, by 
NHES:2019 undeliverable as addressed (UAA) outcome status: 2019 


UAA outcome status 
Non-UAA address 


Inconsistent UAA address 
Number of Percentage of 


Household-level auxiliary data addresses addresses 
Total 85 100.0 
NHES:2019 screener mailings language 
All bilingual mailings (English and Spanish) 30 35.7 
Mix of bilingual mailings and English-only 
mailings 45 51.2 
All English-only mailings 10 13.1 
Age of head of household! 
18-34 5 4.8 
35-44 10 10.7 
45-54 15 15.5 
55-64 5 3.6 
65 and older 15 16.7 
Missing 40 48.8 
Gender of head of household! 
Male 30 33.3 
Female 20 23.8 
Missing 35 42.9 
Education of head of household! 
Less than high school credential 15 16.7 
High school credential 15 19.0 
Some college 15 16.7 
Bachelor's degree 10 9.5 
Graduate degree 5 4.8 
Missing 30 33.3 
Race of head of household! 
White, non-Hispanic 25 27.4 
Black, non-Hispanic 15 16.7 
Hispanic 15 17.9 
Other race, non-Hispanic 5 3.6 
Missing 30 34.5 
Household income! 
Less than $50,000 35 39.3 
$50,000-$74,999 10 11.9 
$75,000-$99,999 10 9.5 
$100,000 or higher 10 14.3 
Missing 20 25.0 


See notes at end of table. 
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Table 7.1. Number and percentage distribution of auxiliary data for 
nonrespondent addresses sampled for observation component, by 
NHES:2019 undeliverable as addressed (UAA) outcome status: 2019— 


Continued 
UAA outcome status 
Inconsistent UAA address Non-UAA address 
Number of Percentage of | Number of Percentage of 
Household-level auxiliary data addresses addresses addresses addresses 
Household flagged as having children! 
Yes 5 7.1 140 23.9 
No 80 92.9 450 76.1 
Number of adults in household! 
1 adult 50 60.7 330 56.4 
2 adults 15 15.5 130 22.1 
3 adults or more # # 10 2.4 
Missing 20 23.8 110 19.2 
Phone number available! 
Yes 35 41.7 360 61.5 
No 50 58.3 230 38.5 
Route type! 
Street 35 41.7 370 62.1 
High rise 50 58.3 220 37.9 
Dwelling type! 
Single-unit 30 34.5 340 57.7 
Multi-unit 55 65.5 250 42.3 
Home tenure! 
Own 30 34.5 290 49.9 
Rent 25 31.0 200 33.1 
Missing 30 34.5 100 17.0 
Urbanicity! 
Urban 40 45.2 270 45.5 
Suburban 40 47.6 290 48.9 
Rural £ £ 10 1.0 
Town 5 6.0 30 4.6 
Region! 
Northeast 5 4.8 80 13.9 
South 55 63.1 250 41.9 
Midwest 25 29.8 170 29.2 
West # # 90 14.9 
Race/ethnicity stratum? 
25% or more Black 30 33.3 150 26.0 
40% or more Hispanic 20 22.6 170 29.0 
Other 35 44.0 270 45.0 


See notes at end of table. 
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Table 7.1. Number and percentage distribution of auxiliary data for 
nonrespondent addresses sampled for observation component, by 
NHES:2019 undeliverable as addressed (UAA) outcome: 2019— 


Continued 
UAA outcome status 
Inconsistent UAA address Non-UAA address 
Number of Percentageof | Number of Percentage of 
Household-level auxiliary data addresses addresses addresses addresses 
Tract poverty rate? 
Less than 20% 50 59.5 390 66.6 
20% or more 35 40.5 200 33.4 
Percent of households in Census block that 
include a child? 
First quartile 30 33.3 160 26.5 
Second quartile 20 21.4 120 20.2 
Third quartile 20 26.2 140 23.9 
Fourth quartile 15 19.0 170 29.4 
Percent of persons in Census block that speak 
a language other than English2 
First quartile 10 14.3 50 8.5 
Second quartile 15 15.5 100 16.1 
Third quartile 20 26.2 180 30.9 
Fourth quartile 35 44.0 260 44.5 
Percent of persons in Census block without a 
high school diploma or the equivalent2 
First quartile 15 20.2 140 23.9 
Second quartile 20 21.4 110 18.3 
Third quartile 20 23.8 130 21.7 
Fourth quartile 30 34.5 210 36.0 
Low Response Score? 
First quartile 5 8.3 110 19.0 
Second quartile 10 14.3 100 16.2 
Third quartile 15 20.2 130 21.9 
Fourth quartile 50 57.1 250 42.9 
Residential high-speed internet per 1000 
households* 
600 or less 25 28.6 140 23.6 
601-800 20 25.0 220 36.7 
801 or more 40 46.4 230 39.7 


See notes at end of table. 

#Rounds to zero. 

+Reporting standards not met. There are too few cases for a reliable estimate. 

1These characteristics are based on variables available on the NHES:2019 sampling frame. 

2These characteristics are based on American Community Survey (2013-2017) five-year estimates. Cases in the first quartile were those with the lowest 
prevalence of the characteristic in question and those that are in the fourth quartile are those with the highest prevalence. 

3The Low Response Score is a derived variable that identifies block groups with characteristics associated with low mail return rates to the 2010 Decennial 
Census. A higher score corresponds to a lower expected mail return rate. This variable was not available for a very small number of cases, and these cases 
have been excluded from this analysis. 

4This characteristic is based on tract-level estimates of Internet penetration provided by the Federal Communications Commission (FCC). 

NOTE: Addresses that ended up responding to NHES:2019 were excluded from this analysis. Addresses that had UAA outcomes for all the NHES:2019 
screener mailings were not eligible for inclusion in the qualitative nonresponse study. Sample sizes are rounded to the nearest 5 (some but not all UAA 
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addresses) or 10 (non-UAA addresses). Percentages are rounded to one decimal place but have not been changed to reflect sample size rounding. Details 
may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Surveys Program (NHES), 2019; and U.S. 
Department of Commerce, American Community Survey (ACS), 2013-2017 and Decennial Census, 2010; and Federal Communications Commission (FCC), 
2017. 


We also compared the distribution of the observed variables collected during the qualitative 
nonresponse study between inconsistent UAA and non-UAA addresses, as summarized in 
table 7.2. Inconsistent UAA addresses were less likely than non-UAA addresses to be fully 
observed, less likely to be occupied residential units, and more likely to be apartments. 


Table 7.2. Number and percentage distribution of observed characteristics of 
nonrespondent addresses sampled for observation component, by 
NHES:2019 undeliverable as addressed (UAA) outcome status: 2019 


UAA outcome status 
Inconsistent UAA 


address Non-UAA address 
Number of Percentage Numberof Percentage 
Observed characteristics addresses of addresses addresses of addresses 
Observation outcome! 
Observed 50 58.3 450 76.1 
Partially observed, could observe the exterior of the 
multi-unit building not the sampled unit 20 25.0 100 16.8 
Not observed, could not access address 5 3.6 20 2.6 
Not observed, could not locate address 5 7.1 20 2.9 
Not observed, other reason 5 6.0 10 1.7 
Residential occupancy status? 
Occupied residential unit 30 45.7 400 72.6 
Residential unit, could not determine occupancy 
status 30 44.3 140 25.8 
Vacant residential unit 10 10.0 10 1.5 
Vacant lot # # $ $ 
Commercial business # # # # 
Structure type? 
Single-unit 15 21.0 260 49.7 
Attached 10 16.1 70 12.8 
Apartment 40 61.3 200 37.1 
Could not determine + $ $ $ 
Mail access type* 
Mail slot or mailbox attached to the home 10 17.7 200 37.3 
Mailbox at the end of the driveway, across the street, 
or at the end of the road 15 24.2 160 30.7 
Mailbox, slot, or room in multi-unit building 15 24.2 80 14.9 
No mailbox or slot in view 10 16.1 40 7.5 
Could not determine 10 17.7 50 9.6 


#Rounds to zero. 

+Reporting standards not met. There are too few cases for a reliable estimate. 

1Partially observed addresses are those where the observer was able to observe the exterior of the multi-unit building, but was not able to enter the 
building to observe the interior entry to the sampled unit. Addresses that could not be accessed are those where observers could not get close enough to 
confirm whether the address existed (for example, if the address was ina gated community). Addresses that could not be located are those where the 
observer was able to access the location where the address should have been, but the observer could not find any evidence that the address existed. Some 
addresses could not be observed for another reason (for example, the observer did not feel safe exiting the car or the sampled unit is hidden by a long 
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private driveway). The rounded number of eligible addresses with some but not all UAAs is 85, and the rounded number of eligible addresses with no UAAs 
is 590. 
2Vacant residential unit includes: (1) temporarily vacant addresses where the unit had a for sale or for rent sign or a lock box and there was no evidence of 
current occupants, (2) seasonally vacant addresses where the unit was located in a resort area and was well-maintained enough to suggest that it had 
tenants during other parts of the year, but there were no evidence of current occupants, and (3) permanently vacant addresses where the address 
appeared uninhabitable (e.g., boarded windows/doors, holes in the walls or roof), was condemned or was under construction. Residential occupancy status 
observations were not collected for addresses that could not be observed (e.g., cannot locate the address). The rounded number of eligible addresses with 
some but not all UAAs is 70, and the rounded number of eligible addresses with no UAAs is 550. 

3Attached structures include duplexes, townhouses, and rowhouses. Apartments include low-, medium-, and high-rise apartments. Structure type 
observations were not collected for addresses that could not be observed (e.g., cannot locate the address) or were observed to be nonresidential or 
temporarily or permanently vacant. The rounded number of eligible addresses with some but not all UAAs is 60, and the rounded number of eligible 
addresses with no UAAs is 530. 
4A few cases where mail was received in more than one way were categorized under "mail slot or mailbox attached to the home" since all the ways mail 
was received fit that category. Mail access type observations were not collected for addresses that could not be observed (e.g., cannot locate the address) or 
were observed to be nonresidential or temporarily or permanently vacant. The rounded number of eligible addresses with some but not all UAAs is 60, and 
the rounded number of eligible addresses with no UAAs is 530. 
NOTE: Addresses that ended up responding to NHES:2019 were excluded from this analysis. Addresses that had UAA outcomes for all the NHES:2019 
screener mailings were not eligible for inclusion in the qualitative nonresponse study. Sample sizes are rounded to the nearest 5 (some but not all UAA 
addresses) or 10 (no UAA addresses). Percentages are rounded to one decimal place but have not been changed to reflect sample size rounding. Details 
may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Surveys Program (NHES), 2019. 


7.3. Quality of Auxiliary Variables on the Frame 


This section focuses on comparing the auxiliary variables available on the sampling frame 
with information that was collected in the field. This evaluation is important because frame 
variables are used for sampling and are increasingly incorporated into NHES recruiting 
strategies. The first part of this section discusses the agreement of the sampling frame 
variables and the observation data. The second part examines the agreement rate between 
the sampling frame variables and the household demographic information collected within 
the qualitative interviews. The final part discusses the characteristics of addresses that were 
missing information on the sampling frame, based on information that was collected as part 
of the observations or interviews. 


7.3.1. Agreement Between Observations and Frame Variables 


Observers collected information about four characteristics that were also available on the 
NHES sampling frame: structure type, residential occupancy status, presence of children, and 
household income. The purpose of this analysis is to understand whether the information on 
the sampling frame is consistent with what observers saw in the field. When interpreting the 
results of these comparisons, it is important to keep in mind that there are limitations to 
human coding, such as human error or limited information being available in the yard or on 
the exterior of the home to use for making an observation. The frame information also has 
similar limitations. In addition, because the observations were collected several months after 
the sampling frame was drawn (in September 2018), it is possible that the occupancy status 
and the characteristics of the household members truly did change during that time. 
However, extensive disagreement between observations and frame data could be considered 
a potential indicator of a problematic frame variable and may raise questions about our 
confidence in using frame variables for targeted materials and procedures. 


The agreement rate between the two sources was calculated as the percentage of addresses 
for which the observation and sampling frame provided equivalent information. The analysis 
for each variable was limited to addresses for which the characteristic in question was both 
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available on the sampling frame and observable during the study.2? Addresses where the 
observation outcome was “could not determine” for a particular characteristic were also 
excluded from the analysis of that variable. While the frame included two categories for 
structure type (single-unit and multi-unit,) the observations further split multi-unit 
addresses into attached units (e.g., townhouse, rowhouse) and apartments. For the purposes 
of this analysis, we assumed that attached units were classified as multi-unit structures on 
the frame. 


The analysis was conducted separately for nonrespondent addresses and inconsistent UAA 
addresses based on the hypothesis that these two address types would have different 
agreement rates for sampling frame variables. As shown in figure 7.3, the agreement rate 
varied considerably across the four frame variables. Among nonrespondent addresses, the 
agreement rate was very high for occupancy status and structure type, but it was lower for 
the presence of children and household income. A somewhat different pattern was observed 
for inconsistent UAA addresses. Here, the agreement rate was high for occupancy status, 
structure type, and the presence of children, but it was lower for household income. The 
remainder of this section examines each of these agreement rates in more detail. 


Figure 7.3. Agreement rate between frame and observation variables for 
nonrespondent addresses and inconsistent UAA addresses sampled for 
observation component, by selected characteristics: 2019 
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1Vacant addresses include those that are seasonally, temporarily, and permanently vacant. 

2For observed data, observers were asked to provide their best estimate of whether the address’s household income was in the bottom third, middle third, or 
top third compared to other households across the United States. For frame data, the bottom third includes those addresses with household incomes less 
than $50,000; the middle third includes those addresses with household incomes of $50,000 to $99,999; the top third includes those addresses with 


30 The percentage of cases that were otherwise eligible to be included in this analysis but were missing data for at least one of the 
variables used to calculate the agreement rate (and hence excluded) was: 0 percent for presence of children, 1 percent for structure type, 
8 percent for household income, and 28 percent for occupancy status. The higher missing rate for the occupancy status analysis was 
driven by the fact that observers often were unable to conclusively determine occupancy status for apartments. 
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household incomes of $100,000 or more. These cutoff points were based on a combination of the distribution of household income in the United States in 
2019 and the pre-existing income ranges available on the NHES sampling frame. 

NOTE: The agreement rate shows the percentage of addresses observed to have the same characteristic found on the frame. Each analysis in this table is 
limited to cases that had data available from both sources. Nonrespondent addresses are those that did not end up responding to NHES:2019 and did not 
have any undeliverable as addressed (UAA) NHES:2019 mailing outcomes. The rounded number of nonrespondent address cases is 540 for structure type, 
410 for occupancy status, 440 for presence of children, and 480 for household income. Inconsistent UAA addresses are those that did end up responding to 
NHES:2019 and had some but not all their NHES:2019 mailings returned as undeliverable. The rounded number of inconsistent UAA cases is 70 for structure 
type, 40 for occupancy status, 40 for presence of children, and 50 for household income. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Surveys Program (NHES), 2019. 


We next assessed whether the agreement rate varied based on (1) how the address was 
classified on the frame or (2) how the observer had classified it. As shown in table 7.3, there 
tended to be variation in the agreement rates for nonrespondent addresses based on these 
classifications. The inconsistent UAA address results are shown in table A.7.5 in appendix A. 


e Occupancy rate: Both the frame and observers classified nearly all the 
nonrespondent addresses as occupied. Therefore, there were not enough vacant 
cases available to discuss whether the agreement rate differed by occupancy status. 
For the few cases that the frame classified as occupied but the observers did not agree, 
the observer noted the presence of lock boxes and “For Sale” or “For Lease” signs, that 
the address was in a resort community, or that it was under construction. For the few 
cases that the frame classified as vacant, but the observers concluded were occupied, 
the observer noted the presence of cars in the driveway, satellite dishes, yard tools, 
or people entering the sampled unit. 


e Structure type: Much of the disagreement for this variable appeared to be about 
whether an address was an attached structure. The frame variable for structure type 
did not include a separate “attached unit” category; therefore, for the purposes of this 
analysis, we assumed that attached units were classified as multi-unit structures on 
the frame. While there was agreement between the frame and the observers for 
nearly all the addresses that observers classified as single-unit structures or 
apartments, the frame and observers only agreed 33 percent of the time for the 
addresses that observers classified as attached units. 


e Presence of children: Observers were significantly more likely to agree with the 
sampling frame for addresses that the sampling frame classified as having no children 
(or being unknown) than they were for addresses that the sampling frame classified 
as having children. Similarly, for addresses where the observers concluded that 
children lived there, the frame only indicated that 40 percent of them included 
children. 


e Household income: The agreement rate was higher for addresses the sampling 
frame classified as in the middle third than it was for those the sampling frame 
classified as in the bottom third or top third. However, the agreement rate was lower 
for addresses that the observers classified as in the middle third than it was for 
addresses that they classified as in the bottom or top thirds. In general, the lack of 
agreement appeared to be driven by observers being more likely than the sampling 
frame to classify an address as having an income in the middle third of the population. 
Based on notes the observers provided in the observation instrument, they used a 
variety of techniques to assess household income, such as drawing on their 
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knowledge of the area; referring to the size, age, or upkeep of the homes in the area; 
or evaluating the type of cars parked in the area or on the upkeep of the yard or 
common areas. A few observers noted uncertainty about their income classification 
and may have defaulted to the middle category because of this uncertainty. If 
observers opted to use the middle third category in lieu of using the “could not 
determine” option when they were unsure of how to classify the household income 
of an address, there would be a higher level of uncertainty in that category. This is 
one potential explanation for why there would be a lower agreement rate compared 
to the more extreme top and bottom categories where observers likely were more 
certain of their household income determination. 


Table 7.3. Agreement rate between frame and observation variables for 
nonrespondent addresses sampled for observation component, by 
selected characteristics: 2019 


Selected characteristics Agreement rate Chi-square statistic 
Occupancy status 
Total 96.3 t 
Occupancy status on frame! Tt 
Vacant? + 
Occupied 98.0 
Occupancy status observation? rT 
Vacant2 $ 
Occupied 98.2 
Structure type 
Total 88.8 t 
Structure type on frame! 27.82 * 
Single-unit 82.8 
Multi-unit 97.3 
Structure type observation? 243.63 * 
Single-unit 97.8 
Attached? 33.3 
Apartment® 95.6 
Presence of children 
Total 66.7 t 
Presence of children on frame! 111.76 * 
Yes 29.7 
No or unknown 82.0 
Presence of children observation? 37.32 * 
Yes 40.4 
No or unclear 73.9 


See notes at end of table. 


144 


Chapter 7. Quality of the Sampling Frame 


Table 7.3. Agreement rate between frame and observation variables for 
nonrespondent addresses sampled for observation component, by 
selected characteristics: 2019—Continued 

Selected characteristics Agreementrate Chi-square statistic 

Household income® 


Total 45.5 t 
Household income on frame! 31.60 * 
Bottom third 47.4 
Middle third 62.6 
Top third 27.3 
Household income observation? 67.42 * 
Bottom third 69.2 
Middle third 28.5 
Top third 53.7 


tNot applicable. 

+Reporting standards not met. There are too few cases for a reliable estimate. 

*p<0.05 

1These characteristics are based on variables available on the NHES:2019 sampling frame. 

2For frame data, vacant addresses were those flagged as being vacant. For observed data, vacant addresses include those that were seasonally, 
emporarily, or permanently vacant. 

3These characteristics are based on address observations collected as part the qualitative nonresponse study. 

4Attached structures include duplexes, townhouses, and rowhouses. Attached structures were assumed to be classified as multi-unit structures on the 
rame. 

5Apartment structures include low-, medium-, and high-rise apartments. Apartment structures were assumed to be classified as multi-unit structures 
on the frame. 

6For frame data, the bottom third includes those addresses with household incomes less than $50,000; the middle third includes those addresses with 
household incomes $50,000 to $99,999; the top third includes those addresses with household incomes of $100,000 or more. These cutoff points were 
based ona combination of the distribution of household income in the United States in 2019 and the pre-existing income ranges available on the NHES 
sampling frame. For observed data, observers were asked to provide their best estimate of whether the address's household income was in the bottom 
hird, middle third, or top third of as compared to other households across the United States. 

NOTE: The agreement rate is the percentage of addresses observed to have the same characteristic as found in the frame. Each analysis in this table is 
limited to cases that had data available from both sources. Rounded number of eligible cases is 540 for structure type, 410 for occupancy status, 440 
for presence of children, and 480 for household income. Addresses that ended up responding to NHES:2019 and addresses that had at least one 
undeliverable as addressed (UAA) NHES:2019 mailing were excluded from the analysis. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Surveys Program (NHES), 2019 


Finally, we conducted subgroup analyses of the agreement rate among nonrespondent 
addresses to determine whether certain address characteristics were associated with higher 
or lower rates of agreement. Using nearly the same methods as described earlier in this 
report, we conducted both bivariate and multivariate analyses (see tables D.9 and D.10 in 
appendix D).3! As summarized in exhibit 7.2, the multivariate analyses did not result in 
consistent patterns in terms of the address characteristics that predicted the agreement 
rate.32 Section D.2.3 in appendix D provides more information about the results of these 
analyses. We did not conduct subgroup analyses for inconsistent UAA addresses due to the 
small number of such cases. 


31 For all agreement rate subgroup multivariate analyses discussed in section 7.3.1, one notable difference from the earlier multivariate 
analyses is that the frame variable that measured the observation outcome in question was not included as an independent variable in 
the regression. For example, the presence of children variable from the frame was not included in the presence of children agreement 
rate multivariate analysis. The goal of these analyses was to determine whether other address characteristics affect the accuracy of these 
indicators on the frame. 

32 We do not show the results for the observability model here because the results of this model should be interpreted with caution. 
Several independent variables needed to be dropped from the model because they perfectly predicted the outcome. The magnitude of the 
differences between subgroups was generally very small, and the agreement rate was 90 percent or higher for all subgroups. 
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Exhibit 7.2. Address characteristics that were significant predictors of agreement 
rate in multivariate logistic regression, by observation variable 


| Observation Address characteristics 


variable 

Structure type e Region: higher agreement rate for addresses in the South and Midwest than for 
addresses in the Northeast and West 

e Tract poverty rate: higher agreement rate for addresses in high-poverty tracts than 
for other addresses 


Presence of e Head of household level of education: higher agreement rate for addresses whose 

children head of household had a graduate degree than for other addresses whose head of 
household had lower levels of education 

Household e Head of household level of education: higher agreement rate for addresses whose 

income head of household had a high school credential or less as their highest level of 
education than for other addresses whose head of household had higher levels of 
education 


e Home tenure: higher agreement rate for addresses inhabited by renters than for 
addresses inhabited by the homeowner 
e Tract poverty rate: higher agreement rate for addresses in high-poverty tracts than 


for other addresses 
SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Surveys Program (NHES), 2019. 


7.3.2 Agreement Between Interview Responses and Frame Variables 


At the end of the interview, interviewers asked participants a series of demographic 
questions. Information about four of these characteristics—Hispanic ethnicity, Spanish- 
speaking status, household income, and number of adults—was also available on the 
sampling frame. This section assesses the extent of agreement between the responses to 
those questions and the information available on the sampling frame. The analysis for each 
sampling frame variable was limited to addresses for which the characteristic in question 
was both available on the sampling frame and from an interview.33 Addresses where the 
interview participant declined to provide information for a particular characteristic were 
also excluded from the analysis of that variable. Subgroup analyses were not conducted due 
to the relatively small number of interview participants. 


The way in which the characteristics were collected was not always perfectly aligned in the 
two data sources. 


e Hispanic households: For the interview variable, Hispanic households are those 
where the participant indicated that he or she was Hispanic during the interview. For 
the sampling frame variable, Hispanic households are those where the sampling 
frame indicated that the head of household was Hispanic. The interview participant 
may not have been the head of household, and this may be a driver of inconsistency 
in some Cases. 


33 The percentage of cases that otherwise eligible to be included in this analysis but were missing data for at least one of the variables 
used to calculate the agreement rate was: 0 percent for Spanish-speaking household, 15 percent for Hispanic household, 16 percent for 
number of adults, and 20 percent for household income. 


146 


Chapter 7. Quality of the Sampling Frame 


e Likely-Spanish-speaking households: For the interview variable, Spanish-speaking 
address are those where either (1) the interview was conducted fully or partially in 
Spanish or (2) the participant reported that Spanish was the language spoken most 
often by the adults in the household. For the sampling frame variable, Spanish- 
speaking addresses are those that met all of the criteria used to identify likely 
Spanish-speakers for a targeted mailings experiment conducted in NHES:2019: (1) 
the household is in a tract where 10 percent or more of households have Spanish as 
the primary language and are “limited English speaking,” (2) the household is flagged 
on the sampling frame as having a Hispanic surname, and (3) the household is in a 
tract with 40 percent or more Hispanic persons. 


e Household income: Both the interview and sampling frame variables were 
categorical variables; for the purposes of this analysis, we collapsed the categories for 
both variables into three categories: bottom third (less than $50,000), middle third 
($50,000 to $99,999), and top third ($100,000 or more). There was a one-dollar 
discrepancy in the categories of the two income variables.34 


e Number of adults: The sampling frame included a variable indicating the number of 
adults that lived in the household. During the interview, participants indicated how 
many people lived in the household and how many of them were age 20 or younger; 
the difference between these two numbers was considered to be the number of adults 
living in the household. The inclusion of 19- and 20-year-olds in the “child” category 
for interview responses may be a factor in the agreement rate. 


In addition, the gap in time between when the interview data were collected and when the 
sampling frame was drawn allows for the possibility that some of the characteristics of the 
household members may have changed. Both factors may contribute to the agreement rates 
reported here. 


As shown in figure 7.4, the agreement rate varied considerably across the four sampling 
frame variables included in this analysis. While the agreement rate was relatively high for 
Hispanic household and likely-Spanish-speaking household, it was rather low for household 
income and number of adults in the household. The remainder of this section examines each 
of these agreement rates in more detail. 


34 For the collapsed three-category variable used for this analysis, the categories were less than $50,000, $50,000-99,99 and $100,000 or 
higher (which was consistent with the frame categories). However, the categories for the interview variable were $50,000 or less, 
$50,001-$100,000 or $100,00 or higher. The discussion of this analysis uses the frame variable categories for simplicity. 
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Figure 7.4. Extent of agreement between interview-gathered and frame 
characteristics of interviewed addresses, by selected characteristics: 2019 
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household” 


Selected characteristics 


1For the frame characteristic, Hispanic households are those where the sampling frame indicated the head of the household was Hispanic. For the interview 
gathered characteristic, Hispanic households are those where the interview participant reported that he or she was Hispanic. 

2For the frame characteristic, likely Spanish-speaking households are those that meet at least one of these criteria based on auxiliary data available on or 
appended to the sampling frame: (1) the household is in a tract where 10 percent or more of households have Spanish as the primary language and are 
“limited English speaking,” (2) the household is flagged on the frame as having a Hispanic surname, and (3) the household is in a tract with 40 percent or 
more Hispanic persons. For the interview-gathered characteristic, likely Spanish-speaking households are those where the interview was conducted fully or 
partially in Spanish or where the interview participant reported that the adults in the households speak Spanish. 

3The income ranges included on the sampling frame and those asked about in the interviews differed very slightly. 

NOTE: Each analysis in this table is limited to cases that had data available from both sources. Rounded number of eligible cases is 85 for likely Spanish 
speaking, and 70 for Hispanic household, number of adults and household income. Details may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Surveys Program (NHES), 2019. 


We next assessed whether the agreement rate varied based on (1) how the address was 
classified on the frame or (2) how the interview participants described themselves. As shown 
in table 7.4, there tended to be variation in the agreement rates based on these 
classifications. The most notable differences are summarized below. 


e Hispanic households: The agreement rate between interview responses and frame 
data was lower for addresses that the sampling frame classified as being Hispanic 
than it was for addresses that the sampling frame did not classify as being Hispanic. 


e Likely-Spanish-speaking households: While there was almost perfect agreement 
between interview responses and frame data for the addresses the sampling frame 
classified as not being likely Spanish-speakers, the agreement rate was considerably 
lower for addresses that the sampling frame did classify as likely Spanish-speaking. 


e Household income: The agreement rate was considerably lower for addresses 


classified by the sampling frame as being in the middle third than it was for addresses 
in the top or bottom thirds. 
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e Number of adults: Agreement with the frame was much more likely when the 
interview participant reported living in a single-adult household than it was when the 
participant reported living in a two-adult household. There were not enough 
interview participants that reported being in a household with three-adult household 
to report the agreement rate for those interview participants. 


Table 7.4. Agreement rate between interview-gathered and frame characteristics 
of interviewed addresses, by selected characteristics: 2019 


Selected characteristics Agreement rate 
Hispanic household! 

Total 86.1 
Hispanic household on frame? 

Yes 72.7 

No 92.0 
Self-reported Hispanic household? 

Yes 80.0 

No 88.5 
Likely Spanish-speaking household* 

Total 75.3 
Likely Spanish-speaking household on frame? 

Yes 42.9 

No 98.0 
Self-reported likely Spanish-speaking household? 

Yes 93.8 

No 71.0 
Household income> 

Total 45.6 
Household income on frame? 

Less than $50,000 58.8 

$50,000-$99,999 26.1 

$100,000 or higher 45.5 
Self-reported household income? 

$50,000 or less 58.8 

$50,001-$100,000 30.0 

$100,001 or higher 35.7 
Number of adults 

Total 42.5 
Number of adults on frame? 

1 34.0 

2 52.9 

3 or more 70.0 


Self-reported number of adults? 


1 85.7 
2 22.5 
3 or more 36.8 
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1For the frame characteristic, Hispanic households are those where the sampling frame indicated the head of the household was Hispanic. For the interview 
gathered characteristic, Hispanic households are those where the interview participant reported that he or she was Hispanic. 

2These characteristics are based on variables available on the NHES:2019 sampling frame. 

3These characteristics are based on self-reports provided by interview participants. In the small number of cases where more than one household member 
participated in the interview, the reported characteristics are those reported by the primary interview participant. 

4For the frame characteristic, likely Spanish-speaking households are those that meet at least one of these criteria based on auxiliary data available on or 
appended to the sampling frame: (1) the household is in a tract where 10 percent or more of households have Spanish as the primary language and are 
“limited English speaking,” (2) the household is flagged on the frame as having a Hispanic surname, and (3) the household is in a tract with 40 percent or 
more Hispanic persons. For the interview-gathered characteristic, likely Spanish-speaking households are those where the interview was conducted fully o1 
partially in Spanish or where the interview participant reported that the adults in the households speak Spanish. 

5The income ranges included on the sampling frame and those asked about in the interviews differed very slightly, as shown in the income range labels usec 
in this table. 

NOTE: The agreement rate is the percentage of addresses where interviewer-gathered characteristics are the same as found in the frame. Each analysis in 
this table is limited to cases that had data available from both sources. Rounded number of eligible cases is 85 for likely Spanish-speaking household, and 7(€ 
for Hispanic household, number of adults and household income. Details may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Surveys Program (NHES), 2019. 


7.3.3 Characteristics of Addresses That are Missing Frame Information 


The final section of this chapter provides information about the characteristics of addresses 
that are missing information on the sampling frame. For sampling frame variables that have 
a “missing” category and were collected during the qualitative nonresponse study (either via 
observations or interviews), we present the percentage distribution of these characteristics 
and compare it to the distribution among addresses that were not missing that information 
on the sampling frame. Ultimately, this is limited to household income (collected in both the 
interview and the observations) and number of adults (collected in the interviews only). 
Because of the smaller number of cases that are missing sampling frame information, this 
analysis combines nonrespondent and UAA addresses. The results of this analysis are shown 
in table 7.5. 


Table 7.5. Number and percentage distribution of selected address characteristics, by 


frame missing status: 2019 
Frame missing status 


Available on frame! Missing on frame! 
Number of Percentage of Number of Percentage of 
Selected address characteristics addresses addresses addresses addresses 
Household income2 
Bottom third 200 29.8 40 34.3 
Middle third 320 48.8 50 44.8 
Top third 90 14.2 10 8.6 
Could not determine 50 7.2 10 12.4 
Household income? 
Less than $50,000 35 42.5 5 100.0 
$50,001-$100,000 20 25.0 # # 
$100,001 or higher 15 17.5 # # 
Refused 10 15.0 # # 
Number of adults? 
1 20 26.3 5 80.0 
2 40 50.0 ch £ 
23.8 # # 


3 or more 20 


#Rounds to zero. 
Reporting standards not met. There are too few cases for a reliable estimate. 


1Frame missing status was determined separately for each analysis based on whether or not a case was missing information for the frame variable that is the focus 
of that analysis. For the household income analyses, the frame missing status is based on whether household income is available on the frame. For the number of 
adults analysis, the frame missing status is based on whether the number of adults is available on the frame. 


2This characteristic is based on household income observations. 
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3This characteristic is based on self-reports provided by interview participants. 

NOTE: The rounded number of eligible cases for observed and self-reported characteristics is 760 and 85, respectively. Sample sizes are rounded to the nearest 10 
(observed characteristics) or 5 (self-reported characteristics). Percentages are rounded to one decimal place but have not been changed to reflect sample size 
rounding. Details may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Surveys Program (NHES), 2019. 


Household Income 


All the addresses that completed an interview and were missing household income on the 
sampling frame reported during the interview that their household income was less than 
$50,000. In comparison, 43 percent of addresses that were not missing this information on 
the sampling frame reported that their household income was in that range. However, 
household income information was missing for a relatively small number of interviewed 
addresses; therefore, this result should be interpreted with caution. 


Household income was also collected as part of the observations. Here, the distribution of 
observed household income was relatively similar among addresses for which this 
information was available on the sampling frame and those that were missing it. About a 
third were observed to have income in the bottom third of the population and about a half in 
the middle third. The remainder were in the top third or could not be determined. 


Number of adults 


Almost all the addresses that completed an interview and were missing the number of adults 
on the sampling frame reported during the interview that there was 1 adult in the household 
(80 percent). In comparison, only about a quarter of the addresses that were not missing this 
information on the sampling frame reported 1 adult in the household (26 percent). However, 
again, information on the number of adults was missing for a relatively small number of 
interviewed addresses; therefore, this result should be interpreted with caution. 
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Chapter 8. Summary and Conclusions 


This chapter starts with a summary of the key findings from each chapter. It then explores 
whether we find evidence in the qualitative nonresponse study findings that supports the 
person-level theories and societal-level drivers of nonresponse discussed in chapter 1. The 
final section reviews the overarching findings of this study, as well as suggestions for next 
steps in this research agenda and the potential implications for future NHES administrations. 


8.1 Key Findings 


The first section of the chapter summarizes the key findings for each of the preceding results 
chapters. 


8.1.1 Nonrespondent Characteristics and Attitudes: Interview Findings 


Chapter 3 provided a detailed exploration of interview participants’ characteristics, 
attitudes, and beliefs. The participants in this study came from a variety of backgrounds and, 
importantly, shared many of the demographic characteristics found to be drivers of 
nonresponse in prior NHES administrations. Their life experiences, beliefs, and attitudes 
often varied by educational attainment, race/ethnicity, and Spanish-speaking status, as well 
as by household attributes such as the presence of children. Taken together, their stories 
offer rich and complex details that help contextualize who they are and how they see and 
experience the world. Below, we summarize some of the key themes related to participants’ 
characteristics and attitudes. 


Busyness. A common thread among many participants was that they felt their lives were 
very busy. More than half of the participants talked specifically about being extremely busy. 
Across all job types, many people worked long hours. One in five participants reported 
having erratic schedules, either at work or while juggling work and other responsibilities, 
such as school or family. When not working for pay, many participants were caregivers. 
Participants from households with children reported being busy more often than those 
without children. 


Community and civic engagement. Most participants felt a sense of belonging to their 
community, primarily through knowing people in the community and having neighborly 
interactions and involvement in children’s school activities. However, some participants felt 
a lack of belonging or connection, mainly due to being new to the area or being an introverted 
or private person. Though many participants thought voting was important, some— 
including both voters and nonvoters—had doubts about whether their vote counted or was 
effective. 


Privacy. Almost all participants discussed privacy at some point during the 
interview, although how they defined privacy and their level of concern about it varied. 
Participants generally defined privacy as (1) protecting personal information (i.e., 
confidentiality or anonymity); or (2) maintaining distance or boundaries between 
themselves and others. About one in six participants were extremely concerned about 
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privacy and took a variety of measures to protect it, including not using social media or cell 
phones, not using banks or credit cards, or burning their mail. At the same time, many people 
believed that there was no such thing as privacy—their information was already freely 
available. 


Government. Most of those who shared their views on the federal government had negative 
views, voicing a variety of concerns including the belief that government was too intrusive, 
concerns about money and corruption, and a feeling that the government was not responsive 
to the general population. When discussing access to and collection of data by the 
government, several participants said they believed that the government already had access 
to their information or that they were skeptical of what the government would do with the 
data it collected. 


8.1.2 Nonrespondent Characteristics and Attitudes: Observation Findings 


Observers also collected observations related to the characteristics and attitudes of the 
individuals living at nonrespondent addresses. They noted the presence of children in just 
under a quarter of the addresses and evidence of privacy or security concerns for just under 
a third of them. For most other characteristics of interest (e.g, community involvement, 
pride in education), relatively few observed nonrespondent addresses that had indicators of 
them visible from the exterior of their home. As an exception, decorative items and indicators 
of enjoying outdoor living were somewhat more commonly observed (each for about a 
quarter of observed nonrespondent addresses). Although still relatively uncommon overall, 
this suggests that some households do place decorative or functional items outside of their 
homes—but it is rarer for these items to provide information about the characteristics or 
interests of the people living there that can be tied back to hypothesized drivers of survey 
response. We also consistently found that indicators of nonrespondent characteristics were 
less likely to be observed for apartments than for single-unit or attached structures. 


8.1.3 Receiving and Processing Mail 


Chapter 4 discussed participants’ behaviors and attitudes related to receiving and 
processing mail, including the results of a mail sorting activity. For most interview 
participants, receiving and processing mail was a routine activity to which they gave little 
thought. Less than 1 in 10 expressed frustration with the process. When they did so, it was 
usually around the volume of junk mail they received. 


Mail retrieval. While many participants checked their mail daily, about a third did so once 
a week or less. Those who did not check every day tended to check the mail only when they 
were expecting certain items to be delivered. Responsibility for checking the mail varied; in 
particular, households with more than two adults were less likely to have a specific 
household member who was responsible for checking the mail. 


Mail access type. Just over a third of observed nonrespondent addresses had a mail 
receptacle attached to their home, and about another third had a mailbox nearby (e.g., end 
of the driveway, across the street). About 15 percent received mail through a mailbox, slot, 
or room in a multi-unit building. Mail access type was highly correlated with structure type, 
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with apartment addresses being more likely than single-unit or attached addresses to 
receive their mail in a centralized location, like a mail room. About 8 percent of addresses 
had no mailbox or slot in view. For the remaining 10 percent, the observer was not able to 
determine the mail access type because he or she could not gain full access to the property 
or building; this was much more likely to occur for apartments than for single-unit or 
attached structures. 


Challenges with mail delivery. About a third of the interview participants discussed 
challenges with mail delivery, such as receiving mail addressed to a different address or not 
receiving expected mail items. Some renters noted that the property owner used their 
address as his or her permanent address; therefore, when mail was addressed to the 
household, they tended to assume it was for the owner and not for them. 


Mail sorting process. Participants tended to sort their mail into one of three categories: 
urgent; junk (mailings that were neither expected nor important); and gray zone (mailings 
where relevance and importance were not immediately clear). Participants’ mail sorting and 
processing behavior varied across four domains: (1) whether they took a break between 
checking and sorting; (2) what types of mail they sorted first; (3) how often they opened junk 
or gray-zone mail; and (4) how long they kept junk or gray-zone mail at their residence. 


Factors influencing engagement with mail. Interview participants completed an activity 
during which they reviewed and sorted a prepared mail bundle as if it was their regular mail 
delivery. Except for those who tended to open all or most of their mail, participants tended 
to scan the envelope for clues that would help them determine a course of action. Factors 
mentioned by participants that influenced whether they would engage with a piece of mail 
included (1) recipient name (e.g., mail that felt more personalized was more likely to be 
opened), (2) sender name (e.g., mail from familiar or trusted senders was more likely to be 
opened), and (3) envelope features (e.g., larger envelopes were more likely to be opened). 
However, no single feature consistently garnered the same reaction from all participants. 
Nor did an identical feature across mailings always prompt the same reaction from a single 
participant. 


While most participants spoke about the importance of having mail directly addressed to a 
member of the household, their choices around engagement were instead influenced by 
numerous factors. Importantly, participants noted that these processing behaviors are 
embedded in the context of their daily lives, meaning decisions may change from mailing to 
mailing based on other demands on their attention and time. 


8.1.4 Experiences with and Opinions About Surveys 


Chapter 5 discussed interview participants’ prior experiences with and opinions about 
surveys, including the NHES. Nearly all participants had participated in a survey in the past. 


General attitudes toward surveys. Participants’ general opinions about surveys varied, 
with approximately a third expressing positive views, a third expressing negative views, and 
another third expressing neutral views. Those who expressed positive views felt that their 
participation helped improve products and services that participants use regularly, 
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contributed to the greater good, or benefitted them personally (e.g., through cash 
incentives). Participants who had generally negative views toward surveys shared that they 
were too busy or had too little time to participate in surveys, felt overwhelmed by the 
frequency of survey solicitations, did not think their participation made a difference in the 
grand scheme of things, or had concerns over sharing their personal information. 


Factors influencing whether participants respond to survey requests. Participants 
were more inclined to participate when they felt connected to the survey topic in some way, 
when they trusted the survey sponsor, when an incentive was offered (although some did 
react negatively and with suspicion toward incentives), when the survey was short, or when 
the survey was presented in a format that made participation more convenient (e.g., 
accessible, user-friendly, and secure). Most participants indicated a preference to complete 
surveys online, provided that they felt the web platform was secure. 


Engagement with and reactions to the NHES:2019 screener mailings. Interview 
participants completed an activity where they were shown the NHES:2019 screener mailings 
and asked whether they remembered them; if so, what they had done with them; and what 
aspects of the mailings had either resonated with them either when they received the 
mailings or during the interview. About three-fourths of the interview participants 
remembered receiving at least one of the NHES:2019 screener mailings. Four-fifths of the 
participants who remembered at least one mailing also opened at least one. Overall, about 
three-fifths of participants opened at least one mailing. Among participants who opened at 
least one mailing, just over half actively decided not to respond, about a fifth saved the 
mailings for later but never responded, and about a quarter ended up responding to the 
survey. 


Participants had a wide range of reactions to the mailings. For some, factors unrelated to the 
NHES design—such as being extremely busy—affected their response to the mailings. For 
others, general “rules” to which they tended to abide in their lives, such as opening all mail 
they receive or not doing any surveys, guided their reaction to the NHES survey mailings as 
well. In addition, the participants discussed a range of general-design and mailing-specific 
features of the mailings that influenced how they reacted to them either when they originally 
received them or when they reviewed them during the interview. Most of these reactions can 
be generally grouped into three factors: (1) the perceived importance of the mailings (e.g., 
government affiliation, FedEx delivery), (2) the perceived relevance of the survey (e.g., 
whether there were children in the household), and (3) the perceived burden or 
intrusiveness of the survey request (e.g., how long they expected the survey to take, whether 
they felt comfortable completing surveys online). 


8.1.5 Nonrespondent Typologies 


Chapter 6 discussed efforts to group participants into typology groups based on their 
classification on several factors identified as key themes in participants’ behaviors and 
attitudes in chapters 3 through 5. Participants were ultimately placed into seven groups; the 
defining characteristics of each group are summarized below. 


155 


Chapter 8. Summary and Conclusions 


Group 1: Late respondents. This group consisted of all interview participants whose 
household responded to the NHES after the fourth screener package. In general, they were 
open or moderately open to doing different types of surveys. They appeared to be less 
concerned than final nonrespondents about privacy. What kept most of them from 
responding to the NHES earlier seems to be related to demands on their time. They reported 
having erratic schedules due to work and family obligations more often than final 
nonrespondents did. 


Group 2: Not enough time. Everyone in this group described themselves as being very busy; 
many had extreme demands on their time, and about one in three talked about being 
completely exhausted. Several explicitly said that they did not have time to take surveys. In 
general, they had relatively open or positive attitudes toward surveys and were not 
extremely concerned about privacy. 


Group 3: Negative attitudes toward the federal government. All participants in this 
group had negative attitudes toward the federal government. Over half described themselves 
as private, the highest of any typology group. But almost none stated that they believed the 
government already had access to their data. Although they were reluctant to participate in 
a government data collection, almost all of them had positive attitudes about 
nongovernmental surveys. Upon opening the NHES mailings, this group had the highest 
percentage of participants who explicitly rejected the survey request. 


Group 4: Federal government already has my information. None of the participants in 
this group saw the purpose of or the urgency in completing the NHES, as they believed that 
the federal government already had access to their data. They believed that the government 
should make use of that before asking them to complete a survey. They also tended not to be 
open to nongovernmental surveys. Their opinions about the federal government also were 
more mixed than the negative attitudes seen in Group 3. They talked about being busy, but 
they did not discuss having erratic work schedules or high levels of exhaustion as frequently 
as Group 2. 


Group 5: Not relevant to me. Everyone in this group believed that they did not need to 
complete the NHES because K-12 education was not relevant to them. All participants but 
one did not have children living with them. This group had the fewest number of people who 
discussed being extremely busy or having erratic work schedules. Most said they were very 
open to completing surveys, but they also were the most likely to report survey fatigue. 
Unlike other groups, they did not often express their opinions about the federal government. 
Among those who remembered at least one NHES mailing, they were less likely than most 
other groups to open the mailings. 


Group 6: Multiple barriers. Participants in this group reported experiencing multiple 
barriers to completing the NHES; each had a combination of factors that seemed to influence 
their lack of response. These participants would likely be the hardest to convert to 
respondents. They tended to hold negative opinions toward surveys and to not be open to 
participating in them. They had privacy concerns, distrusted the government, and juggled 
many time commitments. Most believed that the NHES’s focus on children and education 
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made it not relevant to them. Over half reported challenges with mail delivery. Nevertheless, 
this group had one of the highest rates of recall of the NHES mailings. Most of the participants 
in this group who opened the mailings ended up rejecting the survey request. 


Group 7: Less likely to recall NHES mailings. Everyone in this group checked their mail 
frequently, but just under half of them remembered at least one NHES mailing. This group 
also had the highest percentage of participants who said they tend to open most of the mail 
they receive; however, among those who remembered at least one NHES mailing, this group 
was the least likely to report opening one. About half talked about challenges with mail 
delivery. Other than their lower rate of recall and higher rate of mail delivery issues, the 
other defining characteristic of the participants in this group is that they did not have strong 
opinions or extreme life experiences for the other factors used to create the typology groups. 


Demographic characteristics of typology groups. There were some patterns in terms of 
the characteristics of the participants that fell into the different typologies. In terms of 
characteristics collected during the qualitative nonresponse study, household income, the 
presence of children, and educational attainment varied across many typologies. There was 
also some evidence that membership in the typology groups was associated with certain 
frame variables, such as head of household age and education, household income, and the 
extent of missing data on the frame appear to demarcate certain typologies from others. 
Sometimes the characteristics collected during the study and those available on the frame 
were in agreement about the primary characteristics of typology groups. However, in other 
cases, the two sources provided different pictures of a group’s characteristics. This suggests 
potential challenges for using frame variables to target typology groups in future NHES 
administrations. 


8.1.6 Quality of the Sampling Frame 


Finally, chapter 7 presented a series of analyses aimed at better understanding the quality 
of the NHES:2019 address-based sampling frame. 


Addresses that should not have been on the sampling frame. Two of the observation 
outcomes had the potential to provide insight into whether there might be some addresses 
on the sampling frame that should not have been included: observability and residential 
occupancy status. Observers were able to conduct a full or partial observation for nearly all 
the nonrespondent addresses sampled for the observation component of the study (93 
percent of nonrespondent addresses). Not being able to locate the address was a rare 
outcome (3 percent of nonrespondent addresses). Observers also determined that nearly all 
the observed nonrespondent addresses were residential units (98 percent). It was very rare 
for observers to determine that an observed address was a vacant residential unit, vacant 
lot, or commercial business. 


Addresses with UAA outcomes. We compared the characteristics of addresses with 
inconsistent NHES:2019 UAA outcomes to those without UAA outcomes. This analysis 
suggested that addresses with UAA outcomes are more often (but not always) “problematic” 
addresses. During the address observations, they were less likely to be fully observed, less 
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likely to be determined to be occupied residential units, and more likely to be determined to 
be vacant residential units or residential units with unknown occupancy status. The few 
addresses that were observed to be vacant lots were all those that had inconsistent UAA 
outcomes. 


Quality of auxiliary variables on the sampling frame. We determined the agreement rate 
between the sampling frame and data collected during the qualitative nonresponse study for 
variables that were available from both sources. Although there is potential for error in both 
data sources, low agreement rates may suggest that certain frame variables be incorporated 
into the survey design (e.g., for targeting materials) with caution. Ultimately, the agreement 
rates for nonrespondent addresses ranged considerably. The frame and the observers 
agreed most of the time for occupancy status and structure type (96 percent and 89 percent, 
respectively). The agreement rates were slightly lower for Hispanic ethnicity (86 percent 
agreement with self-reports), for Spanish-speaking status (75 percent agreement with self- 
reports), and for the presence of children (67 percent agreement with observations). And 
the agreement rates were relatively low for household income, (46 percent agreement with 
observations and with self-reports) and for the number of adults in the household (43 
percent agreement with self-reports). 


Finally, we explored the characteristics of addresses that were missing information for 
variables on the frame. All the addresses that completed an interview and were missing 
household income on the sampling frame reported during the interview that their household 
income was less than $50,000, and most reported that there was one adult living in the 
household. 


8.2 Theories of Nonresponse 


As noted in chapter 1, several of the person-level theories and societal-level drivers of 
nonresponse that have been discussed in the survey methodology literature were 
considered in both the development of the study design and the analysis of the results. In 
this section, we return to these theories and drivers; for each, we provide a short overview 
and summarize the relevant themes in the interview participants’ comments. Ultimately, we 
find support for all the theories and drivers—that is, at least some interview participants 
made comments that support each of them. Because of the semistructured nature of the 
interviews, we cannot conclude whether one theory in particular was supported to a greater 
extent than the others. However, the findings of this study largely support the existing 
literature on survey nonresponse. 


8.2.1 Person-Level Theories 


These theories outline attempts to explain social psychological influences on sample 
members’ decisions about whether or not to respond to a survey. They focus on the 
individual relationship the sample member has with the survey request, the perceived value 
of the request, and the meaningfulness to the individual or the individual’s self-perception. 
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Social exchange theory. This theory suggests people make decisions about their social 
behavior based on a cost-benefit analysis (Blau 1964); for example, sample members may 
be more likely to complete a survey request if the perceived rewards of participating exceed 
the perceived costs of doing so (Dillman et al. 2014). When speaking about surveys in 
general, interview participants noted that both the potential for personal gain (e.g., receiving 
incentives, learning more about the survey topic) and the opportunity to contribute affected 
whether they respond to survey requests. In addition, the feeling that participating in a 
survey would not benefit them—or could even cause them harm—was cited by some 
participants as a reason for declining to respond. Participants also indicated that they were 
more likely to complete shorter surveys, those that were in an accessible or user-friendly 
format, and those they received at a time when it was convenient for them to respond. When 
speaking about the NHES screener mailings, a few late respondents noted that seeing the 
note on the third screener letter that the survey may take 3 minutes or less to complete 
helped convince them to do so, while other participants noted that their perception that the 
survey would take too long to complete was a factor in their decision not to respond. The $5 
incentive was also one of the most remembered aspects of the NHES mailings. 


Cognitive dissonance theory. This theory suggests people avoid actions that result in 
cognitive dissonance—that is, they avoid doing things that are inconsistent with their 
perceptions of themselves (Furse and Stewart 1984); for example, sample members may 
decide to respond to a survey to remain in line with their perception of themselves as helpful 
people (Keusch 2015). A few participants noted that they did not respond to the NHES 
because, as a rule, they do not respond to any surveys. More generally, some participants’ 
explanations for why they complete surveys suggested that they feel it is the right thing to 
do or an important thing to do. For example, some participants said that they participate in 
surveys because of the potential for contributing to the greater good or to improve products 
and services. 


Commitment or involvement theory. This theory suggests that commitment and 
involvement are important drivers of a person’s behavior (Becker 1960)—sample members 
may be more likely to complete a survey when they feel committed to or involved with it 
(Elawad et al. 2016). Several participants said they were more likely to complete a survey if 
they were interested in the survey topic—and not to do so if they were not interested in it. 
Feeling that the survey was not relevant to them was the primary response barrier for about 
1 in 6 participants who did not end up responding to the NHES (those in the Not Relevant to 
Me typology group). In addition, several participants noted that their decision about whether 
to participate in a survey depended on whether they knew and trusted the survey sponsor. 
For example, participants’ attitudes toward the government played a role in their reaction 
to government surveys in general and to the NHES more specifically. 


Leverage-saliency theory. This theory suggests that specific survey design features have a 
different amount of /everage on different sample members (Groves et al. 2000; Zhang et al. 
2017; Elevelt et al. 2019). In addition, the amount of leverage a design feature has depends 
on how salient it is made during the survey request. In the qualitative nonresponse study, 
we see evidence for the leverage aspect of this theory; participants had a wide variety of 
reactions to various aspects of the NHES:2019 design. At times, the same feature served as a 
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motivating factor for some participants and a deterrent for others. For example, for many 
participants, features like the $5 incentive or FedEx mailer increased the likelihood of paying 
attention to the survey mailings; however, for some participants, the same elements made 
them suspicious that the mailings were part of a scam. At other times, a design feature was a 
factor in some participants’ response decision but did not seem to matter to others (or at 
least the participant did not raise it as a relevant factor). For example, for those with privacy 
concerns, the request to provide information about their children in the screener decreased 
their interest in responding. But other participants who had children did not mention this as 
an area of concern. 


8.2.2 Societal-Level Drivers 


Additional drivers may have to do with the societal context in which a survey is conducted. 
These may affect the way sample members interpret and respond to a survey request and, 
ultimately, whether they decide to participate. Several of these drivers have been posited to 
be increasing in intensity over time; while the qualitative nonresponse study cannot 
necessarily speak to change over time, interview participants’ comments provide a snapshot 
of the kinds of issues that are currently faced by survey sample members. 


Social integration or isolation. Some researchers have argued that social integration has 
been declining and that this may be a driver in survey nonresponse rates (e.g., Abraham et 
al. 2006; Amaya and Harring 2017; Watanabe et al. 2017). Several participants said that they 
felt connected to their neighborhood or community; for example, some participants 
described knowing their neighbors well and having caring interactions with them, while 
others described close relationships with family members. However, others described 
feeling disconnected. For some, this was because they were new to the area; for others, this 
was a conscious choice to keep to themselves; and for still others, this was a result of not 
fitting in or getting along well with neighbors. 


Privacy concerns. Some researchers have argued that privacy and confidentiality concerns 
have been growing (e.g., Singer and Presser 2008; Kim et al. 2015; Robertson et al. 2018). 
While about half of the interview participants were not highly concerned about privacy, for 
others, it was a relevant factor in their daily life and their survey response decisions. Almost 
all participants took some measures to safeguard their personal data. For example, some 
participants opened even unwanted mail just to check if it contained any personally 
identifiable information (PII); others shredded, or even burned, their mail before discarding 
of it. Some participants expressed concern that the information they provided in surveys 
could be used against them. A few noted that they did not feel comfortable providing 
information about their children in the NHES screener. 


Survey fatigue. Other researchers have noted that the number of survey requests made of 
the population has been growing, and they argue this may be a factor in declining 
participation rates (e.g., Presser and McCulloch 2011; Van Mol 2017). Many interview 
participants who expressed negative views about surveys shared that they found survey 
solicitations to be too frequent and aggressive, particularly online. Some shared that 
receiving too many frivolous survey solicitations contributed to an overall feeling of survey 
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fatigue, which in turn dissuaded them from participating in surveys. Moreover, a few 
participants said that they were turned off by prior experiences with survey requests that 
promised prizes or rewards (e.g., cash prizes) but that turned out to be advertisements. 


Busyness. Finally, it has been posited that individuals are increasingly busy and that this 
may stop them from responding to surveys (e.g., Ingen et al. 2009; Williams and Brick 2018). 
Acommon thread among many interview participants was that they felt their lives were very 
busy. More than half of the participants specifically talked about being extremely busy. Some 
participants indicated that they discarded the NHES mailings without even opening them 
because they were too busy to pay close attention to the mail. Others noted they had saved 
the mailings to address later but forgot to do so because they were too busy; yet others noted 
that, after opening the mailings, they decided they did not have enough time to respond to 
the survey request. Busyness appeared to be the primary response barrier for nearly a 
quarter of the participants that did not end up responding to the NHES (those in the Not 
Enough Time typology group). 


8.3. Considerations for Future NHES Administrations 


In the final section of this chapter, we discuss considerations for next steps in this research 
agenda and the potential implications for future NHES administrations. 


Overall, there was considerable variation in the point at which interview participants 
appeared to have broken off from responding to NHES:2019. This suggests the opportunity 
for changes to multiple components of the mailings to improve response—but also that any 
single change is unlikely to “solve” the entire nonresponse problem. For example, about a 
sixth of participants remembered receiving at least one mailing but decided not to open it; 
this suggests that there may be value in testing changes to the exterior of the mailings. In 
addition, one of the most common outcomes (for nearly half of the participants who did not 
end up responding) was for participants to open the mailings and actively decide not to 
participate; this suggests that there is likely to be additional value in testing changes to the 
messaging and appearance of the screener letters and paper questionnaire to increase 
participants’ likelihood of responding. Finally, about a quarter of interview participants did 
not remember seeing any of the mailings; one potential explanation is that, for some sample 
members, mail may not be the best way to reach them. 


8.3.1 Key Drivers of Nonresponse to Explore in Future NHES Administrations 


Participants’ life experiences, attitudes, and reasons for nonresponse were wide ranging. The 
nonrespondent typology groups presented in chapter 6 provide a starting point for 
understanding the primary drivers of nonresponse. Below we provide suggestions for topics 
to explore in future NHES administrations based on these key drivers. It is important to keep 
in mind that no single driver of nonresponse was mentioned by every participant; moreover, 
for several topics, different subgroups of participants had conflicting opinions on the same 
aspect of the survey design. Therefore, many of the suggestions below should ideally be 
targeted only to those sample members for which they are expected to improve response. In 
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section 8.3.2, we discuss potential considerations for implementing targeted designs in 
future NHES administrations. 


In reviewing these suggestions, it is also important to remember that this study had a small 
sample size and was not designed to be nationally representative of all NHES 
nonrespondents. Therefore, for any proposed changes to the screener materials, focus 
groups, cognitive interviews, or randomized experiments would ideally be conducted before 
using the materials in a full-scale NHES administration. 


Not enough time 


Busyness was a commonly cited concern among interview participants. For just under a 
quarter of the participants who did not respond to the NHES, this appeared to be the primary 
driver of nonresponse (i.e., those in the Not Enough Time typology group). To the extent that 
busyness keeps sample members from checking their mail regularly or being willing or able 
to take the time to participate in any survey, there is little that can be to overcome this 
problem. However, some of the interview participants appeared to fall into more of a gray 
area; for example, saving the mailings to address later or expressing willingness to 
participate if they had more time to do so. To help encourage these sample members to 
respond, there are several modifications that could be explored in future NHES 
administrations. 


e Changes could be made to the appearance of some of the mailings to make them 
stand out when participants are sorting their mail and making quick decisions 
about what to open. For example, some interview participants noted that the 
NHES:2019 mailings that were a unique size or mailing type (e.g., the pressure-sealed 
envelope, larger envelope for packages with paper screeners, and FedEx mailer) 
stood out to them as being important. Additional unique mailing types could 
potentially be incorporated into future NHES administrations, such as USPS Priority 
mailings or square envelopes. However, other envelope features, such as color, font, 
and the use of areal stamp, either elicited mixed reactions from participants during 
the example mail activity—or were not noted at all. These findings - that eye 
catching strategies can undermine the legitimacy of the survey request - support 
earlier research (Dillman et al. 1996; Dillman and Redline 2004). 


e Changes could be made to the mailings—either on the exterior of the mailing or in 
the cover letters—to make it clearer that, for many respondents, the survey does 
not take very long to complete. Although this information is provided in the CAQ 
enclosure, participants’ comments during the interviews suggested that many had 
either skimmed that or did not read it at all. Some participants noted that they 
interpreted the lack of a burden estimate in several of the early screener letters to 
mean that the survey would take a long time to complete. Although the broad 
potential range in survey length (depending on whether or not there is an eligible 
child in the household) makes it challenging to present a single, accurate burden 
estimate for the NHES, there may still be value in attempting to do so. Because the 
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survey can take up to 30 minutes to complete when a child is sampled for a topical 
survey, this change should be tested carefully. 


e Future versions of the NHES screener and topical questionnaires should be as 
short as possible (while still addressing the key goals of the data collection). This 
would minimize the actual burden on respondents, as well as allowing the burden 
estimates in the cover letters to be lower. 


Negative attitudes toward federal government 


Most participants who shared their opinion of the federal government had either mixed or 
negative attitudes. Hispanic participants were particularly likely to be concerned about 
participating in government surveys. In addition, for about a fifth of participants, negative 
attitudes toward the federal government appeared to be the primary driver of nonresponse 
(i.e., those in the Negative Attitudes Toward Federal Government typology group). Although 
they were reluctant to participate in a government data collection, almost everyone in this 
typology group had positive attitudes about nongovernmental surveys and was open to 
completing them. Because the NHES is a government-sponsored survey, the government 
association is deeply entwined in the survey design and may be a challenging barrier to 
overcome. In addition, because several participants said that the government association 
made them pay attention to and open the survey mailings, this is an area in which making 
changes has the potential to backfire; therefore, any changes would ideally be targeted only 
at those sample members who are anticipated to have concerns about participating in 
government surveys. There are two potential changes that could be experimented with in 
future administrations to address these sample members’ concerns. 


e NCES could pursue endorsements from trusted non-profit or community 
organizations and highlight them in the survey materials. This might help 
participants who distrust the government to feel more comfortable completing the 
survey. However, there would be tradeoff between including endorsements from an 
array of organizations that would be relevant to various subgroups of households 
versus the operational implications of having multiple versions of the survey 
materials. Focus groups would likely need to be conducted to confirm that such 
materials would in fact override sample members’ concerns about the survey’s 
government sponsorship. 


e Text could be added to some ofthe screener letters to clarify how the NHES data 
are used. Some interview participants felt that the government should be more 
responsive and accountable to the general public and should invest in making 
changes based on information gathered through surveys. Providing more information 
about how the survey results are used could help address these types of concerns. 


Federal government already has my information 


Some interview participants did not see the purpose of responding to the NHES because they 
thought the federal government already had access to their data. To address this concern, 
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language could be added to the screener letters that makes it clear that the 
government does not have this information and has no other way of getting it. This 
topic is addressed in part in the CAQ enclosure (in response to the question “Why should | 
take part in this study?”) Future versions of the document could also address this concern 
more directly; however, as noted above, the interviews conducted as part of this study 
suggest that not all participants read the CAQ enclosure. 


Not relevant to me 


Several participants, particularly those who did not have children in the household, indicated 
that they did not respond because they did not think the survey was relevant to them. 


e To address this impression, language could be added to the screener letters that 
emphasizes that it is still important for NCES to hear from households without 
children. (Again, although this is mentioned in the CAQ enclosure, it is more likely to 
be noticed if included in the screener letters themselves.)%5 


e Changes could be made to the materials to minimize the immediate association 
with education or children, with the goal of reducing the perception among sample 
members who do not have children that the survey is not relevant to them. Ideally, 
these changes would be made only for households that are not expected to have 
children, as the education focus of the survey appeared to be a motivating factor for 
households that did include children. For example, some participants who did not 
have children noted that the family- and child-focused photographs on the cover of 
the paper screener questionnaire made them think the survey was not relevant to 
them; to address this concern, these images could be removed for households that are 
not expected to have children. 


e Asimplified opt-out version of the screener form could be tested that makes it 
easier for participants without children to respond. For example, the screener 
mailings could include a postcard that sample members who do not have children 
could check a box on and mail back; this could likely be incorporated even in the 
earlier mailings that otherwise do not otherwise have a paper response option. 


Less likely to recall mailings 


Overall, nearly a quarter of the interview participants did not recall receiving any of the 
NHES screener mailings. For many of the participants in the Less Likely to Recall Mailings 
typology group, this appeared to be the primary driver of nonresponse (less than half of them 
recalled the mailings). There are several potential explanations for this outcome. There is 
some evidence that someone else in the household could have retrieved the mailings and 
would have remembered them; for example, the mailing recall rate decreased as the number 
of adults in the household increased. It is also possible these participants’ households may 


35 Several of the suggestions noted in this section involve adding more text to the cover letters. It is possible that making the letters 
longer could backfire, and this potential would ideally be explored in cognitive interviews or focus groups. 
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not have received some or all the NHES mailings.3° For example, participants for whom 
observers had not been able to locate a mailbox were less likely to remember at least one 
NHES mailing. Participants in the Less Likely to Recall Mailings typology group also were 
particularly likely to discuss having mail delivery problems, such as their mail being 
delivered to a neighbor. But most participants in this group also discussed being relatively 
open to participating in surveys. 


These findings suggest there may be a subset of the NHES sample for which mail may not be 
a particularly effective contact method—whether because the reminders are distributed 
among different household members (and, therefore, each household member has fewer 
reminders to respond to) or because the household is less likely to receive the mailings. For 
this group, there may be value in considering the use of alternate modes of contact. For 
example, the auto-call reminder that is typically used in conjunction with the third screener 
package could be moved earlier for this group. Tweaks could also be made to the auto-call 
message to encourage the sample members to call the Census Bureau if they have not 
received the mailings (and provide a phone number in the message for doing so). Other 
phone-based contacts could also be options for this group, such as involving live 
interviewers, who could then administer the screener over the phone or send household 
members an e-mail with the web link and their login credentials. However, to increase the 
likelihood of this being successful, there would ideally be phone numbers available for more 
of the sample members (they were available for about two-thirds of the NHES:2019 sample). 
A larger departure would be to consider sending in-person field staff to these addresses to 
try to reach these households. However, this would likely be very expensive—in part 
because of how widely dispersed the NHES sample is across the United States. To 
incorporate in-person contacts more efficiently, the NHES sample design would likely need 
to change. 


In addition, none of the interview participants in the Less Likely to Recall mailings typology 
group recalled receiving the FedEx mailing—and some mentioned having trouble in the past 
with FedEx deliveries. As a result, it may be preferable to change the FedEx mailing for this 
group to a less expensive U.S. Postal Service (USPS) mailing, or to consider using 
another delivery service company (e.g., DHL, UPS). If sending USPS mailings, the reduction 
in postage costs could potentially free up resources for the kinds of increased phone-based 
contact efforts discussed above. 


As with many of the other suggestions in this section, one of the main challenges is being able 
to identify this group prior to the data collection. This typology group had several self- 
reported demographic characteristics in common and was also somewhat unique in the 
extent to which the self-reported characteristics aligned with the information about the 
household on the sampling frame. This may suggest the potential for targeting this group in 
the future. In addition, addresses that have had UAA outcomes may also be ideal candidates 
for this type of approach; the analyses presented in section 7.2 suggest that UAA addresses 
are more likely to be problematic addresses (e.g., more likely to appear vacant). Efforts could 


36 Another possibility is that these participants simply forgot that they had, in fact, seen the mailings before. But it is not possible to 
evaluate that hypothesis with the available data. 
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be made to attempt, prior to data collection, to predict which cases are most likely to have 
UAA outcomes—or addresses could be assigned to an “other modes” group after having 
received a UAA outcome during data collection. 


Multiple barriers to response 


Finally, interview participants in the Multiple Barriers typology group expressed several 
reasons for their lack of response, including busyness, privacy concerns, and negative 
attitudes toward surveys. These addresses are likely to be the most difficult to convert to 
respondents. The existence of this typology group raises the possibility that there are likely 
some NHES sample members who are not going to respond regardless of what changes 
are made to the design. If such a group could be identified prior to data collection using 
information available on the frame, then fewer resources could be dedicated to this group— 
and those resources could then be reallocated to increasing response among other sample 
members. As discussed in chapter 6, the members of the Multiple Barriers typology group 
did not have very many distinctive demographic characteristics—and some of those that it 
did have were inconsistent between the data collected during the qualitative nonresponse 
study and that the data available on the frame; this may make it challenging to target this 
group in the future. However, future research could be conducted to determine the best way 
to identify this type of participant prior to (or during the early stages of) data collection— 
for example, by reducing the effort for cases with the lowest response propensity scores. 


8.3.2 Targeting Interventions Where They are Likely to Have the Greatest Impact 


Most of the drivers of nonresponse discussed above are only relevant to a subset of 
nonrespondents. And, in some cases, different subgroups had conflicting reactions to an 
aspect of the NHES design. Therefore, many of the proposed changes discussed above would 
ideally be targeted only to those for whom they are expected to improve response. Typically, 
such designs would be targeted based on information available about an address either 
before data collection begins or that becomes available during data collection. A challenge 
faced by ABS surveys, such as the NHES, when trying to use such designs is the limited 
amount of information available about sampled addresses on the sampling frame. There is 
of course some auxiliary data available on the NHES frame, and it may be possible to use 
some of it to target interventions moving forward. However, as seen in the agreement rates 
reported in chapter 7 between the frame variables and the information collected as part of 
the qualitative nonresponse study, the information available on the frame also may not 
always be accurate. For the NHES to be able to target interventions more effectively moving 
forward, it will be important to have more information about sampled addresses. The results 
of the observation component of the study offer some findings about address 
characteristics—but overall, address observations do not appear to provide a full picture of 
nonrespondents’ characteristics and attitudes. Ideally, more auxiliary variables would be 
appended to the NHES frame to provide additional (or more accurate) information 
about the household. 


e Some of the variables that would ideally be incorporated would be improved 
indicators of household members’ demographic characteristics. For example, 
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whether or not there are children in the household impacts whether or not a 
household interprets the NHES as being relevant to them. However, in addition to the 
imperfect agreement rate of the child flag that is currently on the frame with the 
information collected as part of the qualitative nonresponse study, we also see that 
the flag is often not consistent with respondents’ screener responses (Medway et al. 
forthcoming). In NHES:2019, more than half of the households that were flagged as 
having children on the frame did not report that children lived there. In addition, 
about a fifth of those who were not flagged as having children indicated that children 
did in fact live there (this may be due in part to the fact that not being flagged on the 
frame means that either there are not believed to be children living there or the 
information is unknown). 


Other variables that may have value for targeted interventions are those that would 
shed light on participant attitudes that appear to drive nonresponse decisions— 
attitudes toward the government, interest in education, and privacy concerns. 
These kinds of attitudinal variables are not currently available on the NHES frame. 
However, this kind of information may be available from commercial vendors— 
either as individual variables (e.g., a flag indicating a household has a household 
member currently enrolled in school) or as part of consumer segmentation data 
(which use demographic, socioeconomic, and consumer behavior variables to define 
segments and cluster addresses into segments). 


Finally, targeted telephone-based messages may be helpful for reaching certain 
households. However, for NHES:2019, phone numbers were only available for two- 
thirds of the sample. In addition, the interview recruitment phone calls made as part 
of the qualitative nonresponse study suggest that at least some of these numbers are 
not in service or are no longer associated with the sampled address. As a result, it 
would be ideal to have additional, higher quality phone number information. 
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